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Foreword 


This  report  describes  research  conducted  as  part  of  the  Navy  Personnel  Research  and 
Development  Center’s  Distributed  Training  Technology  (DTT)  project.  The  DTT  project  is  part  of 
our  Classroom  and  Afloat  Training  research  program  and  falls  under  the  Education  and  Training 
project  (L1772)  of  the  Navy's  Manpower,  Personnel,  and  Training  Advanced  Development 
Program  Element  (0603707N).  The  work  was  performed  under  the  sponsorship  of  the  Bureau  of 
Naval  Personnel.  The  research  is  evaluating  technologies,  training  strategies,  procedures,  and 
management  methods  to  extend  videoteletraining  (VTT)  beyond  traditional,  lecture-based  courses. 

The  research  investigated  the  feasibility  of  using  videoteletraining  to  deliver  a  Celestial 
Navigation  course.  The  findings  have  direct  implications  for  the  design  of  future  distance 
education  systems  in  the  Navy  and  elsewhere. 

The  recommendations  in  this  report  are  intended  for  use  by  the  Chief  of  Naval  Education  and 
Training  and  Bureau  of  Naval  Personnel  in  developing  policy  for  the  application  of  VTT  in  the 
Navy. 


P.  M.  SPISHOCK 
Captain,  U.S.  Navy 
Commanding  Officer 


MURRAY  W.  ROWE 
Technical  Director 
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Summary 


Problem  and  Background 

Many  Navy  personnel  requiring  training  are  geographically  separated  from  training  resources. 
Videoteletraining  (VTT)  enables  an  instructor  to  teach  multiple  classes  at  different  geographic 
locations.  VTT  has  been  shown  to  be  an  efficient  and  cost  beneficial  way  to  deliver  training,  and  is 
now  in  operational  use  by  the  Chief  of  Naval  Education  and  Training  (CNET)  within  the  CNET 
Electronic  Schoolhouse  Network  (CESN).  VTT  has  been  used  to  deliver  lecture-based  instruction 
and  the  cost  benefits  of  VTT  could  be  extended  if  other  types  of  course  content  could  be  delivered, 
such  as  courses  with  student  laboratories.  A  Celestial  Navigation  course  presented  challenges  for 
conversion  to  the  VTT  format  because  it  contained  several  aspects  that  departed  from  those  found 
in  the  typical  lecture-based  course.  The  course  uses  various  visuals  with  detailed  print  from 
nautical  tables  and  contains  hands-on  laboratories  involving  difficult  computations  and  plotting. 
Concern  also  exists  for  assisting  remote  site  students  in  resolving  problems  with  computations. 

Objective 

The  objective  of  this  research  was  to  evaluate  the  feasibility  of  delivering  a  Celestial 
Navigation  refresher  course  by  video  teletraining  (VTT)  in  terms  of  the  effects  of  VTT  on  student 
performance,  student  attitudes  toward  the  training,  and  instructor-student  interaction. 

Approach 

Three  treatment  groups  were  compared:  (1)  traditional  classrooms;  (2)  VTT  local  classrooms 
with  an  instructor  and  students,  and  (3)  VTT  remote  classrooms  where  students  were  connected  to 
the  local  classroom  by  a  two-way  audio  and  video  VTT  system.  A  total  of  279  students  were  in 
regular  offerings  of  nine  VTT  classes  (135  local  and  66  remote  students)  and  five  traditional  classes 
(78  students).  The  treatment  groups  were  compared  in  terms  of  student  academic  performance, 
student  questionnaire  responses,  and  an  observer  tally  of  instructor-student  interaction. 

Results  and  Conclusions 

Student  academic  performance  was  generally  at  a  high  level  over  three  measures  of 
performance.  There  were  no  significant  differences  between  the  treatment  groups  on  homework 
during  the  week.  There  was  a  small  but  statistically  significant  decrement  in  final  examination 
scores  for  remote  students,  but  there  was  no  significant  difference  between  groups  in  the 
percentage  of  students  passing  the  course.  Statistically  controlling  final  examination  scores  for 
inequities  in  student  characteristics  due  to  different  enrollment  patterns  at  the  sites  showed  an 
adjusted  mean  difference  of  about  4%  between  local  and  remote  sites  when  officer-enlisted  and 
military  seniority  status  were  controlled. 

Student  responses  on  questionnaire  rating  items  were  higher  for  VTT  local  and  traditional 
students  than  for  VTT  remote  students.  However,  the  magnitude  of  differences  between  groups 
was  generally  small.  Ratings  for  all  groups  were  generally  high  and  in  a  positive  direction,  and  the 
observed  pattern  was  typical  of  that  found  in  previous  VTT  research.  The  differences  between 
groups  were  negligible  for  topics  concerned  with  class  participation,  hearing  and  seeing  the 


instruction,  and  instructor  presentations.  Remote  students  gave  slightly  lower  ratings  than  local 
students  on  topics  related  to  getting  assistance  or  attention,  but  did  not  rate  the  difficulty  or  pace  of 
the  course  significantly  different.  Remote  students  were  more  likely  to  agree  that  they  had  less 
access  to  and  interaction  with  the  instructor,  and  to  identify  other  students  as  a  source  of  assistance 
outside  of  class.  However,  remote  students  were  equally  willing  to  take  another  VTT  course  and 
their  preferences  for  a  method  of  instruction  for  this  course  were  not  less  favorable  than  local 
students.  Remote  students  also  strongly  preferred  a  Picture-in-Picture  (PIP)  display  that  showed 
both  the  instructor  and  visuals  during  periods  when  the  instructor  normally  would  have  been  off¬ 
screen.  Facilities  related  questions  indicated  that  table  space  for  publications  and  plotting  was  more 
confined  at  some  sites. 

An  observer's  tally  of  interactions  during  lectures  indicated  that  there  were  adequate  levels  of 
remote  site  student  participation  and  instructor  interaction  with  remote  students.  Interactions  were 
most  commonly  initiated  by  instructors  with  a  mix  of  questions  that  were  left  open  to  any  site, 
directed  at  a  site,  or  directed  at  individual  students  identified  from  a  roster.  This  mixture  allowed 
the  instructor  to  assess  computational  knowledge  in  students  likely  to  respond  when  they  knew  an 
Mswer,  as  well  as  those  individuals  that  were  less  likely  to  participate.  There  tended  to  be  fewer 
interactions  per  hour  associated  with  remote  sites  because  there  were  fewer  students  than  at  the 
local  site.  However,  remote  students  were  not  disadvantaged  when  these  interactions  were 
expressed  in  terms  relative  to  the  number  of  students  (per  student  per  hour).  In  these  terms,  remote 
site  students  received  a  higher  rate  of  instructor  questions  associated  with  their  site  than  did  local 
students.  After  the  first  day  of  class,  student  initiated  questions  increased,  unanswered  instructor 
questions  decreased,  and  instructor  reminders  for  local  site  students  to  use  their  microphones 
declined. 

The  Celestial  Navigation  refresher  course  was  successfully  adapted  for  delivery  by  video¬ 
teletraining.  The  course  has  been  regularly  offered  to  remote  VTT  sites  without  problem  and  is 
accepted  by  students  and  instructors.  The  initial  concerns  with  the  feasibility  of  delivering  the 
course  were  overcome  as  a  result  of  several  course  conversion  techniques  and  the  efforts  of  the 
instructors  to  monitor  remote  site  students.  The  adaptation  of  this  course  and  its  small  print  visuals, 
strip  forms,  and  test  scoring  techniques  provide  an  illustrative  example  of  a  VTT  conversion 
methodology  applicable  to  other  courses. 

Recommendations 

The  following  recommendations  are  for  the  Chief  of  Naval  Education  and  Training  and  the 
CNET  Electronic  Schoolhouse  Network. 

1.  Delivery  of  the  Celestial  Navigation  refresher  course  by  videoteletraining  has  been  suc¬ 
cessful  and  the  use  of  this  method  should  be  expanded  to  other  sites  as  appropriate  to  satisfy 
demand  for  the  course. 

2.  VTT  courses  with  atypical  requirements  such  as  student  laboratories  should  be  given  spe¬ 
cial  attention  to  maintain  a  high  quality  VTT  version  of  the  course.  Such  attention  includes  moni¬ 
toring  remote  site  student  comprehension,  conveying  VTT  lessons  learned  as  remote  site 
facilitators  and  instructors  rotate  in  their  assignments,  and  providing  sufficient  space  for  the  addi¬ 
tional  instructional  materials  found  in  this  and  other  future  laboratory  courses. 
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Introduction 


Problem 

Many  Navy  personnel  who  must  receive  training  are  geographically  separated  from  training 
resources.  An  increasingly  efficient  approach  to  meeting  this  requirement  is  needed  as  the  Navy 
downsizes  and  training  resources  become  constrained.  Videoteletraining  (VTT)  has  been  found  to 
be  an  efficient  and  cost  beneficial  way  to  address  this  issue  because  it  enables  a  single  instructor  to 
teach  multiple  classes  that  are  at  different  remote  geographic  locations.  The  Chief  of  Naval 
Education  and  Training  (CNET)  now  has  VTT  in  operational  use  in  the  CNET  Electronic 
Schoolhouse  Network  (CESN).  This  VTT  system  utilizes  an  interactive  two-way  video  and  audio 
television  system  that  allows  distant  remote  site  students  to  participate  in  the  instruction  originating 
from  a  local  site  where  other  students  are  co-located  with  the  instmctor. 

VTT  has  generally  been  used  for  the  delivery  of  lecture-based  instraction.  Even  with  current 
VTT  technology,  there  is  some  reduction  in  the  quality  of  the  audio  and  video  as  compared  to  live 
instruction;  e.g.,  it  reduces  the  visibility  of  personnel  at  different  classroom  locations  and  also 
reduces  the  ability  of  instructors  and  students  to  interact  as  they  do  in  a  traditional  classroom.  These 
constraints  make  it  more  difficult  to  conduct  training  which  is  not  instructor  centered,  such  as  a 
variety  of  courses  with  hands-on  laboratories  and  learning  environments  that  are  highly  interactive. 
Significant  travel  or  instructor  costs  could  be  avoided  if  such  training  could  be  delivered  via  VTT 
rather  than  in  traditional  classrooms.  A  course  in  Celestial  Navigation  presented  challenges  for 
conversion  to  the  VTT  format  because  it  contained  several  aspects  that  departed  from  those  found 
in  the  typical  lecture-based  course.  The  course  uses  various  visuals  with  detailed  print  from  tables, 
contains  hands-on  laboratories  involving  difficult  computations  and  plotting,  and  there  is  a  concern 
for  assisting  remote  site  students  in  resolving  problems  with  computations. 

Objective 

The  objective  of  this  research  was  to  evaluate  the  feasibility  of  delivering  a  Celestial 
Navigation  refresher  course  by  video  teletraining  (VTT)  in  terms  of  the  effects  of  VTT  on  student 
performance,  student  attitudes  toward  the  training,  and  instructor-student  interaction. 

Background 

Previous  research  and  development  work  has  demonstrated  that  VTT  can  be  an  efficient  and 
cost  beneficial  method  to  electronically  deliver  training  to  remote  Navy  personnel  (Bailey,  Sheppe, 
Kodak,  Kruger,  &  Smith,  1989;  Rupinski  &  Stoloff,  1990;  Rupinski,  1991;  Simpson,  Pugh,  & 
Parchman,  1990, 1991a,  1991b,  1992, 1993;  Stoloff,  1991;  Wetzel,  Radtke,  &  Stem,  1993, 1994). 
Prior  research  on  instractional  television  also  indicates  that  student  achievement  is  not  affected  and 
that  any  initial  unfavorable  attitudes  lessen  as  a  result  of  experience  with  the  medium  (Wetzel,  et 
al.,  1993,  1994).  The  major  cost  benefits  of  video  teletraining  systems  appear  to  be  seen  in 
circumstances  where  travel,  per  diem,  or  duplicated  instmctor  costs  are  avoided  by  usage  that  is 
intense  enough  to  offset  the  costs  of  the  technology.  Courses  that  are  particularly  beneficial  in 
reducing  travel  costs  are  those  with  a  high  student  throughput  and  which  are  short  in  duration  (a 
week  or  less).  Cost  and  efficiency  benefits  have  resulted  from  use  of  the  VTT  system  implemented 
in  the  CESN.  Historical  cost  data  developed  by  the  CESN  from  1989  through  1994  indicate  that 
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the  system  reaches  the  break-even  point  approximately  half  way  through  a  year  (i.e.,  VTT  system 
costs  are  approximately  half  the  travel  and  training  costs  that  are  estimated  to  have  been  avoided). 

The  possibility  of  using  VTT  for  a  wider  range  of  courses  would  extend  the  cost  and  efficiency 
benefits  of  VTT  beyond  the  lecture-based  courses  that  are  typically  delivered  by  VTT.  This  issue 
has  been  addressed  by  experimentally  developing  training  strategies  and  evaluating  technologies 
as  part  of  the  Distributed  Training  Technology  (DTT)  project.  The  purpose  of  this  project  is  to 
extend  the  utility  of  VTT  to  courses  which  present  difficulties  for  this  instructional  medium,  such 
^  those  courses  with  hands-on  laboratories.  The  present  study  extends  other  work  on  adapting 
instructional  content  which  presents  challenges  for  delivery  by  VTT.  Prior  work  in  this  area  with 
Navy  courses  includes  the  hands-on  laboratories  in  a  Damage  Control  Petty  Officer  course 
(Simpson,  et  al,  1992),  leadership  training  involving  small  group  processes  (Simpson,  Wetzel,  & 
Pugh,  1995,  Wetzel,  Simpson,  &  Seymour,  1995),  and  general  guidelines  for  converting  courses  to 
VTT  (Simpson,  1993).  Other  in-progress  work  to  be  reported  later  includes  hands-on  laboratories 
in  a  Fiber  Optic  Cable  Repair  course  and  the  use  of  computers  in  a  Quality  Assurance  course. 

The  Celestial  Navigation  course  differs  somewhat  from  traditional  lecture-based  courses  in 
several  ways  that  made  it  more  difficult  to  convert  to  VTT.  Although  much  of  the  course  is 
instructor-centered  and  involves  lectures,  there  is  also  a  significant  computational  component  of 
the  course  that  contains  materials  and  activities  that  depart  from  the  typical  lecture  based  course. 
There  were  several  concerns  about  the  feasibility  of  delivering  the  course  when  it  was  being 
considered  for  VTT  during  the  time  that  the  San  Diego  CESN  was  being  brought  on-line  at  the 
Fleet  Training  Center  (FTC).  The  course  is  a  difficult  refresher  course  that  assumes  some  prior 
knowledge  of  the  content.  Nautical  publications  with  numerous  small  print  tables  are  shown  by  the 
iristructor  on  an  overhead  projector  which  supports  a  resolution  greater  than  that  obtained  with 
video.  Relative  to  other  Navy  laboratory  courses  with  more  substantial  equipment  and  activities, 
the  Celestial  Navigation  course  could  be  said  to  involve  a  mild  form  of  a  hands-on  laboratory! 
These  hands-on  student  laboratories  involve  the  use  of  four  publications  to  perform  computations 
and  various  small  items  of  equipment,  such  as  used  for  plotting  on  charts. 

Several  adaptations  were  developed  to  allow  the  course  to  go  on-line  as  a  VTT  course  that  went 
beyond  the  typical  need  to  convert  overhead  projector  slides.  For  example,  problems  with  showing 
detailed  small  print  in  publications  were  circumvented  by  developing  special  inserted  graphics,  and 
plastic  overlays  were  developed  so  that  plotting  work  could  be  scored  by  facilitators  at  remote 
sites.  Students  can  show  an  instructor  their  work  and  resolve  difficulties  that  arise  with  their 
computations  in  a  traditional  classroom  where  the  instructor  is  physically  present.  The  interactive 
nature  of  the  aid  provided  to  students  by  instructors  in  traditional  laboratories  is  more  difficult  to 
provide  to  remote  site  students  who  are  accompanied  only  by  the  VTT  facilitator  or  other  students. 
Subject-matter  expertise  is  not  a  requirement  for  VTT  facilitators,  whose  duties  are  to  maintain  the 
remote  classroom,  operate  equipment,  distribute  materials,  and  score  tests.  Thus,  it  was  of  concern 
whether  remote  students  would  be  able  to  get  help  in  performing  computations  when  the  instructor 
could  not  circulate  among  students  and  look  over  their  shoulders  in  the  interactive  way  that 
traditional  laboratories  permit. 

It  was  also  of  interest  whether  several  technologies  could  be  employed  to  assist  in  delivering 
the  course.  A  significant  portion  of  the  course  involves  the  instructor  working  through  a  series  of 
computational  steps  that  are  written  on  a  “strip  form”  template  that  is  shown  to  the  class.  During 
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these  periods  the  VTT  instructor  can  spend  a  good  deal  of  time  off-camera  when  writing  on  strip 
forms  or  displaying  graphics  which  must  be  shown  from  a  video  document  camera  on  the  podium. 
An  alternative  to  this  single  view  was  introduced  by  using  picture-in-picture  (PIP)  technology  and 
a  robot  tracking  camera  to  allow  a  combined  view  of  both  the  strip  form  and  the  instructor  to  be 
shown  simultaneously.  A  questionnaire  was  employed  to  assess  whether  students  found  this 
technology  useful.  Another  questionnaire  was  used  to  assess  any  preference  for  an  electronic 
presentation  program  that  was  adapted  to  show  a  strip  form  in  one  of  the  instructor's  lessons. 

The  primary  issues  in  evaluating  the  feasibility  of  the  VTT  version  of  the  Celestial  Navigation 
course  were  as  follows:  (a)  how  well  do  remote-site  students  perform  academically  in  the  course 
compared  with  students  who  are  in  a  classroom  with  the  instructor,  (b)  what  effect  did  the  VTT 
medium  have  on  student  acceptance  and  attitudes,  (c)  what  effect  did  VTT  have  on  the  level  of 
interaction  and  participation  for  remote  students  compared  with  those  at  the  local  site,  and  (d)  what 
practical  efforts  were  required  to  deliver  the  VTT  course  that  provide  lessons  learned  applicable  to 
other  courses  (e.g.,  conversion  of  materials  to  a  form  suitable  to  VTT). 

Method 

The  overall  plan  for  the  evaluation  study,  treatment  groups,  outcome  measures,  and  a 
description  of  the  course  and  the  methods  used  to  convert  the  course  are  described  below.  The 
outcomes  of  interest  were  a  comparison  of  student  grades,  student  evaluations  of  the  training,  and 
the  level  of  instructor-student  interaction  in  VTT  classes. 

Treatment  Groups 

A  single  independent  variable  (type  of  instruction)  with  three  states  was  used.  This  variable 
consisted  of  three  treatment  groups:  (1)  students  in  a  traditional  classroom  with  an  instructor;  (2) 
a  VTT  local  classroom  where  students  were  also  with  an  instructor,  and  (3)  VTT  remote 
classrooms  where  students  were  connected  to  the  local  classroom  by  a  fully  interactive  two-way 
audio  and  two-way  video  VTT  system.  A  subsample  of  students  were  also  included  in  an 
evaluation  of  picture-in-picture  (PIP)  technology  and  the  use  of  an  electronic  presentation  program 
described  later. 

Outcome  Measures 

Dependent  variables  (outcome  measures  to  assess  the  effect  of  the  independent  variable)  were 
in  three  general  categories:  student  perceptions  of  VTT  and  training  quality,  student  academic 
performance,  and  instructor-student  interaction.  Each  of  these  variables  was  defined  in  terms  of  the 
measures  described  below  and  data  were  gathered  with  the  data  collection  instruments  in 
Appendices  A,  B,  and  C. 

Student  Performance 

Three  aspects  of  student  performance  were  examined:  (1)  homework  percentage  correct,  (2) 
final  examination  percentage  correct,  and  (3)  percentage  of  students  passing  the  course.  Student 
performance  in  this  course  is  assessed  on  two  types  of  work:  in-class  assigned  work  problems  and 
a  practical  final  examination.  The  criterion  for  passing  the  Celestial  Navigation  course  is  75 
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percent  for  both  the  assigned  work  and  the  final  examination.  If  either  of  these  is  below  75  percent, 
the  student  does  not  pass  the  course. 

Assigned  homework  during  the  week  is  generally  performed  in  periods  allotted  during  class, 
after  class,  or  performed  off  site.  The  percentage  correct  over  13  of  these  exercises  is  averaged  for 
the  homework  portion  of  the  course  grade.  The  final  examination  consists  of  a  lengthy  problem 
scenario  that  takes  approximately  five  hours  to  complete.  The  students  begin  the  problem  on  the 
afternoon  of  the  fourth  day  and  complete  the  examination  on  the  final,  fifth  day. 

Student  Questionnaire 

A  44  item  questionnaire  was  developed  for  the  Celestial  Navigation  course  (see  Appendix  A). 
The  questions  were  organized  in  terms  of  the  following  categories:  (1)  instructors,  (2)  learning  & 
classroom  activities,  (3)  training  aids,  (4)  interaction/participation,  (5)  overall  course  rating,  (6) 
background  and  prior  experience  with  the  course  material,  (7)  video-teletraining,  and  (8)  open 
ended  questions  soliciting  comments  on  VTT,  likes  and  dislikes  about  the  course,  and  suggestions 
for  improvements.  Most  of  the  questions  (34  items)  used  a  rating  format  where  agreement  or 
disagreement  with  a  statement  was  rated  on  a  five  point  scale  ordered  as  follows:  [1]  Strongly 
Disagree,  [2]  Disagree,  [3]  Neither  Agree/Disagree,  [4]  Agree,  [5]  Strongly  Agree.  The  remaining 
questions  consisted  of  six  multiple  choice  items,  a  five-point  rating  item  on  the  pace  of  the  course, 
and  four  open  ended  questions,  one  of  which  was  a  second  part  of  a  multiple-choice  item. 

There  were  two  versions  of  the  questionnaire,  one  for  VTT  classes  (44  items)  and  one  for 
fraditional  classes  (38  items).  The  38  item  questionnaire  for  the  traditional  classes  used  the  same 
items  as  the  44  item  questionnaire  except  that  six  items  were  omitted  because  they  related  to  VTT 
and  could  not  be  answered  by  the  traditional  classes  (items  34-37  and  40-41).  Items  38  and  39  were 
slightly  modified  to  allow  administration  in  the  traditional  classes.  Item  38  did  not  have 
facilitator  as  a  choice  for  assistance  provided  to  students,  and  item  39  was  prefaced  by  a  short 
paragraph  describing  VTT  so  students  could  answer  which  method  of  instruction  they  would  prefer 
for  the  course. 

Interaction  Tally 

An  observer  recorded  the  frequency  of  interaction  among  sites  during  the  first  three  days  of  the 
course  for  six  VTT  class  convenings.  The  interaction  tally  was  recorded  for  interactions  that 
occurred  over  the  VTT  network  and  involved  recording  the  frequency  of  events  in  the  categories 
given  below.  The  observer  recorded  interactions  from  a  separate  room  connected  to  the  network 
and  was  not  present  in  either  the  local  or  remote  classroom.  Interactions  had  to  be  related  to  course 
content  and  were  recorded  during  lecture,  but  not  laboratory  sessions.  Start  and  end  times  for  each 
class  period  were  recorded  on  the  tally  form  in  order  to  account  for  variations  of  these  periods  over 
different  class  convenings.  For  each  of  the  first  five  categories  below,  a  check  mark  was  entered  in 
a  box  corresponding  to  whether  it  occurred  at  the  local  site  or  each  of  the  remote  sites. 

1.  Instructor  Question  Open  to  Any  Site:  Instructor  asks  a  question  (and  receives  an 

answer)  in  which  no  site  is  identified  so  that  all  sites  are  free  to  respond  to  the  question. 
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2.  Instructor  Question  Names  a  Site:  Instructor  asks  a  question  (and  receives  an  answer) 
that  identifies  a  site  which  should  answer,  either  the  local  site,  a  specific  remote  site,  or  “any” 
remote  site. 

3.  Instructor  Questions  Specific  Student:  Instructor  asks  a  question  (and  receives  an 
answer)  directed  at  one  student  by  singling  out  the  student  by  name,  title,  or  by  pointing  (a 
class  roster  for  each  site  was  created  during  the  first  morning  of  the  class). 

4.  Student  Initiated  Question:  A  student  initiates  a  question  to  the  instructor,  class,  or 
another  student. 

5.  Student  Conversations  and  Exchanges:  Student  exchanges,  comments,  or  general  con¬ 
versations  involving  continued  back-and-forth  exchanges  on  an  issue,  such  as  to  resolve  a 
problem.  These  were  most  commonly  a  continuation  of  a  student  initiated  question  that 
resulted  in  more  than  one  communication  each  for  the  instructor  and  student,  including  a  sto- 
dent  other  than  the  one  initiating  the  question. 

Several  other  miscellaneous  circumstances  were  recorded  separately  in  addition  to  the  above 
categories.  Categories  6  and  7  were  recorded  for  the  whole  class  and  not  by  remote  and  local  sites. 
Category  8  was  recorded  only  for  local  students. 

6.  No  Answer  to  Question:  There  was  no  answer  from  students  to  an  instructor  question 
which  was  specific  to  the  instructional  content. 

7.  No  Answer  to  Generic  Request  for  Questions:  The  instructor  asked  in  one  way  or  another 
if  there  were  “any  questions”  and  no  response  was  received  from  the  students.  These  questions 
were  generic  queries  that  were  used  to  pause  before  proceeding  with  the  instruction. 

8.  Reminders  to  Local  Students  to  Use  Microphones:  The  instructor  reminded  the  local  San 
Diego  site  to  use  the  push-to-talk  microphones  so  remote  sites  could  hear  questions  and 
comments. 

Subjects  and  Classes  Observed 

VTT  classrooms  were  at  three  locations:  the  originating  or  VTT  local  site  at  the  Fleet  Training 
Center  (FTC)  in  San  Diego,  CA,  and  two  VTT  remote  sites  located  at  the  FTC  Detachment  at 
Treasure  Island,  CA,  and  at  the  Trident  Training  Facility  (TRITRAFAC)  at  Bangor,  WA. 
Traditional  classes  were  conducted  at  the  FTC  in  San  Diego  in  a  conventional  non- VTT  classroom 
designed  for  the  Celestial  Navigation  course. 

Student  grades  were  compared  for  a  sample  of  279  total  students  who  received  similar  training 
materials.  These  students  were  drawn  from  nine  convenings  of  the  VTT  class  (135  local  and  66 
remote  students)  and  from  five  convenings  of  the  traditional  class  (78  students)  conducted  during 
the  same  time  periods  as  the  VTT  classes.  Of  the  nine  VTT  classes.  Treasure  Island  was  a  remote 
site  nine  times  and  Bangor  was  a  remote  site  five  times.  For  the  classes  studied,  the  average  class 
size  was  15.0  students  for  VTT  local,  7.3  for  VTT  remote,  and  15.6  for  the  traditional  classes.  An 
additional  set  of  archival  student  grades  was  examined  for  91  students  in  another  five  traditional 


5 


classes  that  had  used  somewhat  different  instructional  materials  prior  to  the  time  that  the  course 
had  been  converted  to  VTT. 

A  subset  of  the  total  sample  of  279  students  completed  questionnaires  and  were  observed  to 
record  the  interaction  tally.  Questionnaires  were  completed  in  nine  VTT  class  convenings  (128 
VTT  local  and  61  VTT  remote  students)  and  in  three  traditional  class  convenings  (52  students). 
Interaction  tallies  were  recorded  by  observers  for  six  of  the  VTT  classes  (the  local  site  was 
connected  with  two  VTT  remote  sites  for  three  convenings  and  with  only  one  remote  site  during 
the  other  three  convenings). 

All  students  were  enrolled  in  normally  scheduled  class  convenings.  The  typical  officer  student 
was  a  Lieutenant  Junior  Grade  or  Lieutenant  (84%  of  all  officers  were  0-2  and  0-3).  The  typical 
enlisted  student  was  in  the  Quartermaster  rating  and  was  a  Third,  Second  or  First  Class  Petty 
Officer  (82%  of  all  enlisted  were  E-4  to  E-6).  As  discussed  later,  the  enrollment  pattern  of  the  intact 
groups  examined  here  resulted  in  relatively  more  enlisted  than  officer  students  at  remote  sites 
compared  to  the  local  site.  Remote  site  students  were  also  slightly  more  junior  than  those  at  the 
local  site.  The  number  of  students  per  group,  average  rank,  and  the  period  of  the  study  are  given  in 
Table  D-1  of  Appendix  D. 

One  instructor  taught  about  three  fourths  of  the  class  sessions  during  the  study  and  the 
remainder  of  the  sessions  were  taught  by  one  or  two  other  instructors.  The  primary  instructor  for 
VTT  classes  had  previously  taught  the  course  in  the  traditional  manner  (i.e.,  prior  traditional 
classes  identified  in  Appendix  D).  Three  students  in  the  study  later  became  instructors  themselves 
and  taught  portions  of  later  classes. 

VTT  Course  Preparation 

Description  of  Course 

The  five  day  Celestial  Navigation  refresher  course  studied  was  taught  at  the  Fleet  Training 
Center  in  San  Diego  (Course  Identification  Number  J-061-0603).  There  are  typically  15  to  18 
students  per  class.  A  maximum  of  24  students  can  be  accommodated  within  the  VTT  classroom. 
The  course  consists  of  lectures  and  practical  application  computational  exercises.  These 
encompass  celestial  navigation  theory,  time,  sunrise  and  sunset,  moonrise  and  moonset,  star-finder, 
sextant  correction,  latitude  by  Polaris,  local  apparent  noon,  azimuths,  and  celestial  computations 
using  nautical  and  air  almanacs  and  standard  strip  forms  (vertical  forms  used  to  enter  a  sequence 
of  computations).  Students  are  officers  supervising  or  assigned  to  navigational  duties  and  enlisted 
Quartermasters  (QM)  filling  a  billet  as  a  leading  QM.  The  course  is  a  refresher  course  and  assumes 
prior  knowledge  or  practical  experience  in  performing  the  celestial  navigation  computations. 

The  course  involves  lecture  and  laboratory  sessions  over  the  first  four  days.  Laboratory 
sessions  involve  practical  experience  in  performing  computations  and  some  plotting.  Some 
laboratories  are  interspersed  among  lectures  during  the  day,  and  the  first  three  days  end  with  a  two 
hour  laboratory  session  to  complete  the  days  assigned  homework,  some  of  which  is  often 
performed  later  off-site.  A  lengthy  final  examination  begins  on  the  afternoon  of  the  fourth  day  and 
continues  through  midday  of  the  final  fifth  day  of  the  course.  A  42  minute  film  or  videotape  is 
shown  twice:  early  on  first  day  of  instruction  and  again  during  the  fourth  day.  The  film  covers 
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celestial  navigation  concepts  as  part  of  the  “theory”  portion  of  the  course.  It  uses  limited  animation 
and  includes  color  drawings  of  the  relation  of  the  earth,  heavenly  bodies,  and  various  projected 
lines  and  angles.  Students  use  four  navigation  publications  and  a  variety  small  navigational 
equipment. 

VTT  Course  Conversion 

Converting  the  Celestial  Navigation  course  to  VTT  required  efforts  that  exceeded  those  usually 
required  to  convert  a  typical  lecture-based  course.  For  example,  remote  sites  had  to  be  outfitted 
with  numerous  materials  and  equipment  for  use  in  student  laboratories,  there  were  many  visuals 
with  small  print  in  tables  that  required  special  attention,  and  a  method  had  to  be  devised  to  allow 
student  plotting  to  be  scored  at  remote  sites. 

The  following  equipment  and  supplies  were  procured  to  outfit  the  two  remote  sites;  (1)  four 
publications  (two  navigation  handbooks  and  two  nautical  almanacs  consisting  of  tables);  (2) 
parallel  rulers  were  used  in  lieu  of  drafting  machines  used  in  the  traditional  course;  (3)  star  finders; 
(4)  circular  nautical  slide  rules;  (5)  drafting  dividers;  and  (6)  blank  plotting  charts. 

A  number  of  paper  items  had  to  be  duplicated  for  remote  sites.  These  included  homework 
sheets  and  strip  forms  for  13  exercises,  and  various  other  handouts.  A  daily  schedule  of  events  was 
developed  so  that  remote  site  facilitators  could  distribute  materials  to  students  at  the  appropriate 
time.  Remote  site  personnel  were  required  to  maintain  secure  copies  of  the  final  examination  to 
avoid  test  compromise. 

There  were  a  substantial  number  of  transparencies  shown  by  the  instructor  in  the  traditional 
class.  Many  of  these  transparencies  displayed  tables  containing  small  print  and  were  shown  on  an 
overhead  projector  with  a  higher  resolution  than  is  supported  by  video.  These  graphics  are  used  for 
orientation  purposes  so  that  students  can  locate  entries  in  their  own  copy  of  the  relevant 
publication,  such  as  which  page  is  currently  being  used,  various  row  and  column  headings,  and  the 
relevant  entries  within  the  table  on  that  page.  Figure  1  illustrates  a  sample  graphic  using  a 
technique  developed  to  overcome  the  difficulty  with  showing  the  small  print  in  the  tables  used  in 
the  course.  The  critical  portions  of  the  text  have  been  placed  within  “bubbles”  containing  enlarged 
text  and  arrows  point  to  where  that  entry  is  located  on  the  page  (stick-on  labels  with  darkened  edges 
were  easily  adapted  for  this  purpose).  Because  the  print  is  normally  very  small,  the  bubbles 
directed  student  attention  to  the  relevant  entries  that  were  large  enough  to  read.  The  strip  forms 
were  also  enlarged  and  segmented  into  several  pages  because  these  vertical  forms  can  be  as  long 
eis  a  typewritten  page  and  cannot  be  seen  in  their  entirety  over  a  video  system  (an  example  is  shown 
later  in  Figure  2). 

Transparencies  were  converted  to  a  paper-based  form  so  that  they  could  be  presented  on  the 
video  document  camera  stand.  A  landscape  orientation  was  used  for  visuals  as  appropriate  to  the 
3:4  aspect  ratio  of  television.  The  resolution  supported  by  video  required  that  the  visuals  be 
converted  to  contain  fewer  lines  and  words  (pages  with  about  15  lines  and  25-30  characters  per  line 
are  typically  recommended).  A  similar  relative  size  of  the  visuals  on  a  page  was  maintained  so  that 
readjustments  of  the  zoom  lens  on  the  document  stand  would  be  minimized  as  the  instructor  moved 
through  the  slides. 
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POLARIS  (POLE  STAR)  TABLES,  1993 

FOR  DETERMINING  LATITUDE  FROM  SEXTANT  ALTITUDE  AND  FOR  AZIMUTH 
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argument;  a„  a,  are  taken,  without  interpolation,  from  the  second  and  third  tables  with  arguments  latitude  and 
month  respectively,  a,,  a„  a„  are  always  positive.  The  final  table  gives  the  azimuth  of  Polaris. 

Figure  1.  Example  of  revised  instructor  slide  for  small-print 
nautical  publications. 


An  unfortunate  side  effect  of  converting  the  course  to  VTT  Avas  that  the  computational 
problems  themselves  had  to  be  updated.  The  previous  problems  were  based  on  a  prior  year  of  the 
nautical  almanac  which  could  no  longer  be  purchased  for  each  student  at  the  two  remote  sites. 
Consequently  new  computational  problems  for  the  current  year  were  developed  based  on  new 
almanacs  purchased  for  all  sites. 


The  42  minute  16mm  film  used  in  the  course  was  professionally  converted  to  videotape  form. 
The  quality  of  the  film  was  not  materially  affected  and  was  adequate  when  viewed  via  the  digitally 
compressed  video  of  the  VTT  system. 

Remote  site  facilitators  scored  student  final  examinations  using  new  scoring  methods  designed 
to  captured  the  expertise  of  the  instructors.  An  answer  key  for  the  numeric  entries  on  the  test  form 
was  easily  prepared,  but  the  plotting  portion  of  the  test  was  more  problematic.  An  acetate  overlay 
was  developed  for  the  facilitators  to  place  over  a  student's  plotting  chart.  The  overlay  was  marked 
with  the  correct  course,  “fixes,”  and  labeling  required  of  students.  Remote  site  facifitators  for  the 
initial  VTT  class  received  training  from  the  instructors  on  the  course  procedures  during  a  visit  to 
San  Diego. 
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VTT  Classroom  Configuration 

The  training  sites  were  equipped  with  a  fully  interactive  two-way  video  and  two-way  audio 
digital  VTT  system.  Digital  video  was  transmitted  over  terrestrial  T1  lines  at  384  Kbps  using  a 
VTEL  brand  386  MediaMax  Codec  and  Multipoint  Control  Unit.  The  classrooms  were  equipped 
with  a  minimum  of  three  video  cameras  that  could  be  selected  by  a  handheld  remote  or  from  a 
control  panel  at  the  rear  of  the  room.  One  video  camera  on  the  rear  wall  showed  a  view  of  the 
instructor  and  another  pan/tilt  video  camera  on  the  front  wall  to  showed  a  view  of  the  students.  An 
instructor  podium  at  the  front  of  the  room  was  equipped  with  an  Elmo  brand  video  document 
camera  used  to  show  hardcopy  visual  materials.  The  instructor  could  view  a  small  monitor  below 
a  glass  top  on  the  podium  when  positioning  graphic  material  on  the  video  document  camera.  The 
classrooms  were  not  equipped  with  chalkboards  or  whiteboards.  All  students  viewed  40-60”  TV 
monitors  to  see  the  instructional  materials  shown  on  the  video  document  camera. 

Remote  site  students  viewed  all  instruction  on  large  40”  TV  monitors.  Local  site  students  who 
were  physically  present  with  the  instructor  viewed  only  the  instructional  materials  or  remote  site 
students  on  the  monitors.  The  televised  version  of  the  class  viewed  by  remote  site  students 
alternated  between  a  view  of  the  instructor,  the  printed  materials  shown  on  the  document  camera 
stand,  and  occasionally  a  view  of  the  local  students.  Two  large  color  monitors  were  located  at  the 
front  of  the  room,  usually  near  the  center  for  remote  students  and  to  the  side  in  each  comer  for  local 
students.  Remote  sites  were  t5q)ically  configured  so  that  one  front  monitor  showed  the  incoming 
video  and  the  other  showed  the  outgoing  video.  Two  monitors  in  the  back  of  the  room  allowed  the 
instructor  to  view  the  outgoing  video  and  the  incoming  video  from  the  remote  site  that  had  most 
recently  spoken. 

Students  at  all  sites  could  hear  each  other  by  using  a  push-to-talk  microphone  that  was  shared 
by  each  pair  of  students.  Push-to-talk  microphones  were  used  in  order  to  reduce  extraneous  noise, 
such  as  from  private  conversations,  shuffling  of  papers,  or  moving  books.  Pressing  a  microphone 
had  the  effect  of  switching  the  video  to  a  view  of  the  site  using  the  microphone.  Students  were 
instructed  in  the  use  of  the  VTT  system  on  the  first  day  of  class.  Students  in  the  local  classroom 
were  reminded  to  use  the  microphones  so  that  remote  sites  could  hear  them. 

The  VTT  classrooms  accommodated  a  maximum  of  24  students  in  three  rows  of  tables  facing 
the  front  of  the  room.  The  arrangement  of  the  rooms  varied  somewhat  from  site  to  site.  The  local 
San  Diego  classroom  was  equipped  with  12  tables  with  a  short  depth  (24  X  60").  An  aisle  down 
the  middle  of  the  room  separated  the  tables,  each  of  which  was  shared  by  a  pair  of  students.  Remote 
classrooms  had  somewhat  bigger  tables  during  the  study  and  contained  fewer  students  so  that  there 
was  more  room  to  spread  out  student  materials. 

Each  remote  classroom  was  monitored  by  a  facilitator  who  was  not  a  subject-matter  expert  in 
the  course  content.  The  facilitator  was  responsible  for  the  operation  of  the  equipment  from  the  rear 
of  the  classroom,  maintaining  order,  class  rosters,  distributing  and  collecting  class  materials,  and 
scoring  the  homework  and  final  examination. 
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Technology  Evaluation  Questionnaires 


Two  small  evaluations  of  new  technologies  introduced  into  the  course  were  conducted  with  a 
subsample  of  the  students  described  in  the  main  study.  The  value  of  using  two  technologies  was 
evaluated:  Picture-in-Picture  views  and  an  electronic  presentation  program. 

Picture-in-Picture  Evaluation 

The  objective  of  this  evaluation  was  to  assess  the  desirability  of  enhancing  an  instructor’s 
presentation  with  Picture-in-Picture  (PIP)  technology  during  lectures  where  the  instructor's  visuals 
are  shown  extensively.  The  main  video  image  of  an  instructor's  visual  aids  was  accompanied  by  a 
smaller  inset  PIP  panel  which  showed  a  view  of  the  instructor.  The  visual  aids  shown  during  VTT 
classes  are  presented  on  a  video  document  camera  or  copy  stand  and  consist  of  word  displays, 
tables,  drawings,  pictures,  and  writings  or  drawings  performed  by  the  instructor  during  a  class 
session.  The  use  of  PIP  allowed  a  view  of  both  the  instructor  and  his  visuals  to  be  seen 
simultaneously  instead  of  just  one  view  or  the  other.  This  technique  allows  the  instructor  to 
maintain  a  continuous  visual  presence  instead  of  only  being  heard  from  off  screen  while  the  visuals 
are  shown.  However,  this  technique  could  be  detrimental  if  the  PIP  image  were  allowed  to  overlay 
portions  of  the  visual  aids  so  students  cannot  see  relevant  information,  or  if  the  PIP  image  of  the 
instructor  distracted  students  from  observing  relevant  information  in  the  visual  aids. 

Figure  2  depicts  the  screen  layout  that  was  employed  by  using  a  Panasonic  WJ-MX30  video 
mixer  to  create  the  PIP  image  as  a  vertical  panel.  A  strip  form  being  written  on  by  the  instructor  is 
shown  on  the  left  and  the  instructor  is  shown  from  the  waist  up  on  the  right  of  the  screen.  The  PIP 
image  was  consistent  with  the  vertical  orientation  of  the  strip  forms  being  used  and  was 
approximately  30%  of  the  horizontal  width  of  the  screen.  A  ParkerVision  “CameraMan”  robot 
tracking  device  was  used  to  keep  the  image  of  the  instructor  at  the  podium  centered  in  the  PIP 
window.  This  device  tracks  the  instructor  by  following  a  pair  of  infrared  lapel  targets  worn  on  the 
front  and  back  of  the  instructor.  The  pan  and  tilt  of  the  camera  is  continuously  adjusted  to  track  the 
lapel  targets  as  the  instructor  moves.  The  tracking  typically  involves  small  left  or  right  movements 
and  a  constant  offset  setting  allows  the  view  of  the  instructor  to  be  positioned  in  a  predefined  region 
of  the  video  screen. 

The  evaluation  design  involved  presenting  periods  of  instruction  with  PIP  alternately  enabled 
or  disabled,  or  with  PIP  continuously  enabled.  Students  then  completed  a  questionnaire  on  the 
value  of  the  PIP  technique  and  their  preference  for  its  use.  Three  groups  of  students  were  employed 
in  the  evaluation.  Two  groups  of  students  received  alternating  periods  of  instruction  during  which 
PIP  was  either  enabled  or  disabled.  This  “Alternating  PIP”  treatment  was  given  to  37  VTT  local 
and  15  remote  students  during  two  class  convenings.  An  additional  group  of  17  VTT  remote 
students  were  in  a  “Regular  PIP  Use”  condition  during  three  class  convenings  when  the  PIP  was 
used  throughout  the  week. 
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Figure  2.  Picture-in-Picture  (PIP)  display  of  strip  form  and  instructor. 

The  alternating  PIP  treatment  conditions  were  given  to  students  according  to  a  scheme  that 
balanced  the  order  and  exposure  time  of  the  PIP  and  non-PIP  periods  of  instruction.  The  order  of 
the  conditions  was  balanced  over  two  class  convenings  by  rotating  the  conditions  over  the  morning 
and  afternoon  periods  of  the  first  two  days  of  the  classes.  During  the  first  class  convening,  the 
sequence  on  the  first  day  involved  using  PIP  during  the  morning  (PIP)  and  not  using  PP  in  the 
afternoon  (No-PP),  and  on  the  second  day  the  morning  was  No-PIP  and  in  the  afternoon  was  PP. 
During  the  second  class  convening,  the  sequence  on  the  first  day  was  No-PP  in  the  morning,  PIP 
in  the  afternoon,  and  on  the  second  day  PIP  in  the  morning  and  No-PP  in  the  afternoon.  During 
periods  when  PP  was  not  used  the  full  screen  showed  only  the  visuals  or  strip  form  and  the 
instructor  did  not  appear  in  the  right  hand  portion  of  the  screen.  The  tracking  camera  was  used 
during  both  PIP  and  non-PIP  presentations  for  all  treatment  groups. 

A  simple  verbal  announcement  was  given  on  the  morning  of  the  first  day  of  the  class  that  PP 
would  be  used  during  some  class  periods  and  not  others  and  that  a  brief  evaluation  questionnaire 
would  be  given  later.  The  11  item  PP  evaluation  questionnaire  given  in  Appendix  B  contained  10 
items  rated  on  a  five  point  strongly-agree  to  strongly-disagree  scale  and  one  two-choice  item. 
Students  in  the  alternating  PIP  groups  completed  the  PIP  questionnaire  at  the  end  of  the  second  day 
of  the  course. 
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The  PIP  technique  was  implemented  on  a  regular  basis  in  the  course  following  the  data 
collection  with  the  alternating  PIP  conditions.  When  used  as  a  regular  feature  throughout  the 
course,  the  PIP  was  only  shown  on  the  outgoing  video  to  the  remote  students  and  was  not  shown 
on  the  monitors  for  local  students  who  could  see  the  instructor  directly  in  the  classroom.  A  total  of 
17  students  who  received  PIP  on  a  regular  basis  were  administered  questionnaires  on  the  final  day 
of  the  course  during  three  additional  class  convenings. 

Electronic  Presentation  Program  Evaluation 

A  computerized  slide  presentation  program  was  used  to  present  one  of  the  celestial  navigation 
lessons  that  used  strip  forms.  A  Generi  Parametrics  Corporation  VideoShow  device  was  used  for 
this  lesson,  although  any  of  several  other  manufacturers'  software  program  could  have  been 
employed  in  conjunction  with  a  computer-to-video  scan  converter.  The  program  was  used  to 
present  an  electronic  version  of  a  strip  form  which  serially  revealed  each  of  the  entries  that  the 
instructor  would  have  normally  written  as  the  computations  on  the  strip  form  were  performed. 

A  short  five  item  questionnaire  was  administered  to  students  that  asked  them  to  compare  the 
electronic  presentation  program  lesson  with  the  conventional  hand  written  method  used  in  other 
lessons  (see  Appendix  C).  The  same  five  point  agree-disagree  rating  scale  used  in  the  celestial 
navigation  student  questionnaire  was  used  for  four  of  the  items.  Two  items  asked  for  a  rating  of  the 
effectiveness  of  the  hand-written  and  electronic  methods  respectively.  Two  items  asked  for  a  rating 
on  the  readability  of  the  text  with  the  two  methods  respectively.  One  item  offered  a  two-altemative 
choice  between  the  two  methods.  The  questionnaire  was  completed  by  15  VTT  local  students  and 
8  VTT  remote  students  during  one  class  convening. 

Results 

Traditional,  VTT  local,  and  VTT  remote  treatment  conditions  are  compared  below  in  terms  of 
student  performance  in  the  course,  student  responses  on  the  post-course  questionnaire, 
participation  patterns  from  the  interaction  tally,  and  student  responses  on  technology 
questionnaires.  The  number  of  cases  used  for  the  analyses  varied  slightly  over  these  different 
outcome  measures  (cf.  Appendix  D).  The  most  important  comparisons  in  this  data  are  between  the 
two  VTT  groups  because  they  attended  exactly  the  same  classes,  whereas  the  traditional  students 
attended  separately  convened  classes. 

Student  Performance 

Student  performance  in  the  course  was  assessed  in  terms  of  three  measures:  average  percentage 
correct  on  assigned  work  (homework)  during  the  week,  percent  correct  on  the  final  examination, 
and  the  proportion  of  students  passing  the  course.  Figure  3  shows  these  measures  of  performance 
separately  for  officer  and  enlisted  students  (additional  detailed  breakdowns  of  the  results  are  given 
in  Appendix  D,  Tables  D-2,  D-3,  and  D-4).^  Scores  for  279  students  were  analyzed,  consisting  of 

Student  performance  was  also  examined  archivally  for  an  additional  five  traditional  classes  convened  prior  to 
converting  me  course  to  VTT.  These  classes  were  not  included  in  the  analyses  reported  here  because  the  level  of  per¬ 
formance  differed  from  traditional  classes  that  occurred  after  course  materials  were  converted  for  VTT.  Tables  in  Ap¬ 
pendix  D  show  that  final  examination  performance  and  the  percentage  of  students  passing  was  much  lower  in  these 
prior  classes,  possibly  because  the  examination  content  was  more  difficult. 
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140  officer  students  (48  traditional,  76  VTT  local,  and  16  VTT  remote),  and  139  enlisted  students 
(30  traditional,  59  VTT  local,  and  50  VTT  remote). 


Figure  3.  Average  percent  correct  for  student  homework  and  final 

examination  performance,  and  percentage  of  students  passing 
the  course. 


The  analyses  of  student  performance  were  performed  with  each  of  the  three  treatment  groups 
subdivided  into  officer  and  enlisted  student  subgroups.  This  approach  was  necessitated  because  the 
level  of  performance  by  officer  and  enlisted  students  differed  and  there  was  an  unequal  distribution 
of  these  students  in  the  three  treatment  groups.  Enlisted  students  were  much  more  likely  to  appear 
at  remote  sites  than  were  officer  students  (75%  of  remote  site  students  were  enlisted,  whereas  44% 
of  local  and  38%  of  traditional  site  students  were  enlisted).  Over  all  treatment  groups  combined, 
officer  students  performed  at  a  slightly  higher  level  than  did  enlisted  students  (4.2%  higher  on 
homework  and  5.5%  higher  on  the  final  examination).  Thus,  the  officer-enlisted  subdivision  of 
treatment  groups  avoids  the  comparison  of  a  more  equal  mix  of  students  at  the  local  site  with 
remote  sites  that  primarily  contained  enlisted  students  that  scored  slightly  lower  regardless  of  site. 

A  two-way  analysis  of  variance  (ANOVA)  design  with  two  between-subject  factors  was 
employed  to  test  statistical  significance  with  the  homework  and  final  examination  scores.  The  first 
factor  consisted  of  the  three  treatment  groups  (traditional,  VTT  local,  and  VTT  remote),  and 
second  factor  subdivided  these  groups  into  officer  and  enlisted  groups.  This  analysis  provides  a 
main  effect  test  of  significance  for  each  of  the  two  factors  and  a  test  of  whether  the  two  factors 
interacted  as  a  function  of  particular  levels  of  the  factors.  The  number  of  students  passing  the 
course  was  evaluated  by  a  test  of  proportions  expressed  as  Chi  Square  (Fleiss,  1981,  p.l39). 
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Homework 


The  average  percent  correct  for  the  13  homework  exercises  during  the  week  is  shown  in  the  left 
hand  panel  of  Figure  3.  There  were  only  negligible  differences  between  treatment  groups.  The 
largest  group  difference  was  only  1.4%  within  the  enlisted  groups.  Officers  scored  from  3.2%  to 
4.8%  higher  than  enlisted  students.  The  only  significant  effect  revealed  by  an  analysis  of  variance 
was  a  main  effect  difference  between  officer  and  enlisted  students,  F(l,273)  =  25.81,  p  <  .01.  The 
main  effect  for  the  three  treatment  groups  was  not  significant,  F(2,273)  =  0.72,  p  >  .05.  The 
interaction  effect  was  not  significant,  F(2,273)  =  0.36,  p  >  .05,  indicating  a  similar  pattern  among 
treatment  groups  for  both  officer  and  enlisted  students. 

Final  Examination 

The  average  percent  correct  on  the  final  examination  is  shown  in  the  middle  panel  of  Figure  3. 
The  highest  performance  was  found  with  VTT  local  students,  followed  closely  by  traditional 
students,  and  the  lowest  performance  was  with  the  VTT  remote  students.  Hie  largest  difference 
obtained  was  between  the  two  VTT  groups,  where  VTT  remote  was  lower  than  VTT  local  by  4.4% 
for  officers  and  5.1%  for  enlisted  students.  An  analysis  of  variance  indicated  significant  differences 
for  the  treatment  groups  main  effect  F(2,272)  =  6.85,  p  <  .01,  and  for  the  officer-enlisted  groups 
main  effect  F(  1,272)  =  19.34,/?  <  .01.  The  interaction  effect  was  not  significant,  F(2,272)  =  0.05, 
p  >  .05,  indicating  that  the  same  pattern  among  treatment  groups  was  found  among  both  officer  and 
enlisted  students.  Tukey  HSD  tests  indicated  that  the  VTT  remote  group  was  significantly  lower  {p 
<  .01)  than  VTT  local  and  traditional  groups,  which  did  not  differ  significantly  from  one  another. 

An  alternative  analysis  approach  that  could  be  substituted  for  that  given  above  would  be  to 
control  for  the  effect  of  differing  student  characteristics  that  resulted  from  nonrandom  enrollment 
pattern  among  the  sites.  In  addition  to  there  being  an  inequity  in  the  number  of  officer  and  enlisted 
students  among  the  sites,  remote  students  were  also  slightly  more  junior  than  those  at  local  sites 
(Table  D-1  in  Appendix  D).  Final  examination  scores  were  correlated  0.33  with  officer-enlisted 
status  and  0.22  with  increasing  militaiy  experience  (expressed  as  Z-score  military  rank  within  each 
officer  and  enlisted  group).  An  analysis  of  covariance  (ANCOVA)  used  these  two  student  attributes 
as  covariates  to  adjust  the  treatment  group  scores  for  the  different  mix  of  these  student 
characteristics.  The  ANCOVA  also  showed  the  three  treatment  groups  to  be  significantly  different, 
F(2,273)  =  5.62,  p  <  .01.  Based  on  this  analysis,  the  adjusted  group  means  are  93.1%  for 
traditional,  94.3%  for  VTT  local,  and  90.3%  for  VTT  remote.  The  local  and  remote  groups  differ 
by  only  4.0%  when  officer-enlisted  and  military  seniority  are  controlled,  and  would  differ  by  4.8% 
if  only  officer-enlisted  status  were  controlled  (Table  D-3  in  Appendix  D).  Thus,  remote  site  student 
performMce  appears  somewhat  higher  when  adjusted  for  inequities  in  student  characteristics  that 
varied  with  the  pattern  of  enrollment  among  the  sites. 

Percent  Passing  Course 

The  percentage  of  students  passing  the  course  is  shown  in  the  right  hand  panel  of  Figure  3. 
Students  passed  the  course  if  the  homework  and  final  examination  percentages  were  both  75%  or 
more,  and  failed  if  either  score  was  below  this  criterion.  All  officer  students  passed  the  course  in 
all  three  treatment  groups.  The  percent  of  enlisted  students  passing  was  similar  among  the  three 
groups,  which  differed  by  4.6%  at  most.  A  slightly  higher  rate  of  passing  was  shown  by  traditional 
students  (96.6%)  than  either  of  the  two  VTT  groups.  The  VTT  local  (93.2%)  group  passed  the 
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course  at  nearly  the  same  level  as  the  VTT  Remote  (92.0%)  group.  In  terms  of  actual  frequencies, 
the  number  of  enlisted  students  who  did  not  pass  the  course  was  1  of  30  for  traditional,  4  of  59  for 
VTT  local,  and  4  of  50  for  VTT  remote. 

Whereas  all  officer  students  passed  in  all  three  treatment  groups,  only  the  enlisted  student  data 
were  evaluated  with  a  test  of  Ae  proportion  passing  (Fleiss,  1981,  p.  139).  The  proportion  of 
students  passing  in  the  three  groups  did  not  differ  significantly  (x^(2)  =  0.69,  p  <  .05). 

Discussion  of  Performance 

Taken  together,  student  performance  was  generally  at  a  high  level  over  the  three  performance 
measures.  There  were  no  significant  differences  between  the  treatment  groups  for  the  homework 
and  the  percentage  of  students  passing  the  course.  Remote  site  students  performed  significantly 
lower  on  the  final  examination.  However,  the  4%  to  5%  lower  performance  on  this  examination 
was  not  large  and  did  not  significantly  affect  the  number  of  students  passing  the  course. 

A  potential  explanation  for  part  of  the  difference  between  groups  on  the  final  examination  was 
offered  by  instructors.  Informal  observations  suggested  an  occasional  remote  site  student 
performed  poorly  due  to  poor  plotting  practices.  A  little  over  half  of  the  examination  points  are 
awarded  on  the  quality  of  the  plotting  itself  and  the  use  of  standard  conventions,  such  as  whether 
and  how  labels  have  been  written  for  the  course,  speed,  fix,  projected  positions,  lines  of  position, 
changes  in  course  and  speed,  and  the  angle  of  the  labels.  If  at  least  some  of  the  difference  between 
groups  is  on  this  portion  of  the  examination,  then  it  may  reflect  fewer  opportunities  for  remote 
students  to  have  been  corrected  by  the  instructor  during  the  week,  which  is  more  readily 
accomplished  at  the  local  site  when  instructors  circulate  among  students  during  laboratories 
involving  plotting.  If  further  monitoring  of  the  course  shows  this  explanation  to  be  true,  then  it 
would  be  recommended  that  (1)  corrective  guidance  could  be  given  by  the  facilitator  or  redirected 
to  the  instructor  by  viewing  remote  student  work  on  the  video  document  camera,  or  (2)  a  training 
aid  could  be  developed  such  as  a  handout  showing  examples  of  both  errors  and  correct  usages  of 
the  plotting  standards. 

Student  Questionnaire 

Student  evaluations  and  perceptions  of  the  training  and  VTT  were  measured  with  a  post-course 
questionnaire  (Appendix  A)  which  contained  a  series  of  statements  to  be  rated,  multiple-choice 
questions,  and  open-ended  questions.  The  questionnaire  was  completed  by  a  total  of  241  students 
(52  traditional,  128  VTT  local,  and  61  VTT  remote).  These  students  were  drawn  from  five 
traditional  and  nine  VTT  class  convenings.  Students  in  the  three  treatment  groups  completed  a 
common  set  of  questionnaire  items  (1-33,  and  38-39, 42-44).  A  subset  of  items  that  applied  only 
to  students  in  VTT  classrooms  were  not  included  on  the  questionnaire  given  to  traditional  students 
(items  34-37  and  40-41). 

Student  Prior  Experience  and  Background 

Items  3 1-33  asked  the  students  about  their  prior  proficiency  and  type  of  experience  in  manually 
performing  the  celestial  navigation  computations,  and  their  future  use  of  the  skills  from  the  course. 

Item  3 1  asked  students  to  rate  the  statement  “Before  this  course,  I  was  proficient  at  manually 
performing  the  celestial  navigation  computations”  on  a  five  point  scale:  (1)  strongly  disagree,  (2) 
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disagree,  (3)  neither  agree/disagree,  (4)  agree,  and  (5)  strongly  agree.  Student  responses  were 
slightly  below  the  midpoint  of  the  scale,  indicating  slight  disagreement  with  the  statement.  The 
averages  for  traditional,  V  IT’  local,  and  V  IT  remote  were  2.54,  2.46  and  2.88,  respectively.  An 
analysis  of  variance  showed  no  significant  difference  between  these  groups,  F(2,236)  =  2.3l,p> 
.05.  The  correlations  between  responses  to  item  31  and  student  performance  in  the  course  were 
negligible  (the  Pearson  correlation  with  percent  correct  on  homework  and  the  final  examination 
were  0.072  and  -0.013,  respectively,  df  =  237,  p  >  .05). 

Item  32  asked  I  previously  had  experience  in  manually  performing  the  celestial  navigation 
computations  in  this  course”  and  presented  the  following  alternatives:  (1)  No;  (2)  Yes,  on-the-job 
experience,  (3)  Yes,  course  work;  (4)  Yes,  both  on-the-job  experience  and  course  work;  (5)  Other 
(please  explain).  No  responses  to  this  question  indicated  that  a  similar  number  of  students  in  the 
three  treatment  groups  said  they  did  not  have  prior  experience  (17%  traditional,  16%  local,  and 
18%  remote).  A  test  of  these  proportions  of  “No”  responses  was  not  significant  (x^(8)  =  0. 16,'  p  > 
.05)  (Fleiss,  1981,  pl39).  However,  Figure  4  shows  that  there  were  differences  in  the  pattern  of 
responses  of  officers  and  enlisted  students  in  the  other  response  categories.  While  a  substantial 
proportion  of  both  groups  had  both  on-the-job  experience  and  course  work,  many  more  enlisted 
than  officer  students  indicated  only  on-the-job  experience,  whereas  many  more  officer  than 
enlisted  students  indicated  only  course  work.  Combining  the  three  “Yes”  categories,  67.5%  of  the 
officers  and  84.5%  of  the  enlisted  indicated  prior  experience  in  one  or  the  other,  or  both,  of  the 
experience  categories,  a  difference  of  17%. 


Type  of  Prior  Experience  in  Computations 


Officer  Enlisted 


Other 

No 


Both  On-the-Job 
Experience  and 
Coursework 


Coursework 


On-the-Job 

Experience 


Figure  4.  lype  of  student  prior  experience  in  performing  celestial 
navigation  computations. 
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Item  33  assessed  future  use  of  the  course  content.  The  question  asked  “Will  your  duties  or  next 
assignment  directly  involve  application  of  skills  from  this  course”  and  presented  the  following 
alternatives:  (1)  No;  (2)  Yes,  infrequently  for  shipboard  duties;  (3)  Yes,  routinely  for  shipboard 
duties;  (4)  The  course  was  primarily  for  advancement  purposes;  (5)  Other  (please  explain).  A 
similar  pattern  of  response  was  found  for  traditional,  local,  and  remote  sites  and  a  Chi  Square  test 
showed  no  significant  difference  in  the  distribution  of  responses  to  these  categories  for  the  three 
groups  (%^(8)  =  3.9, p  >  .05).  The  category  “Yes,  routinely  for  shipboard  duties”  was  chosen  by  77, 
77,  and  73  percent  of  the  traditional,  local  and  remote  students.  About  three  quarters  of  the  officer 
(73.2%)  and  enlisted  (79.1)  students  indicated  they  would  frequently  use  the  skills  from  the  course. 
Officers  were  slightly  more  likely  to  indicate  that  they  would  infrequently  use  the  skills  (22.8%) 
than  were  enlisted  students  (8.7%). 

The  responses  to  the  three  background  questions  (31,  32,  and  33)  indicated  that  students 
generally  had  the  prior  experience  required  for  this  refresher  course  and  were  taking  the  course 
because  of  the  direct  application  of  the  skills  in  their  shipboard  duties.  The  three  treatment  groups 
were  similar  in  their  prior  experience  and  it  was  not  related  to  their  performance  in  the  course. 

Student  Ratings 

Statements  rated  by  students  were  grouped  in  six  categories;  instructors,  learning  and 
classroom  activities,  training  aids  and  materials,  interaction/participation,  overall  course  rating, 
and  VTT  (items  1-30,  34-37  in  Appendix  A).  Statements  were  rated  on  a  5-point  scale  with  a 
midpoint  of  3  using  the  following  scale  values  and  labels:  (1)  strongly  disagree,  (2)  disagree,  (3) 
neither  agree/disagree,  (4)  agree,  and  (5)  strongly  agree.  Item  30  used  a  different  rating  scale  as 
described  below. 

Figures  5,  6  and  7  show  the  mean  ratings  computed  for  traditional,  VTT  local  and  VTT  remote 
classrooms.  Most  students  gave  positive  ratings  to  the  dimension  being  measured  because  the 
average  rating  for  the  positively  phrased  items  was  above  the  midpoint  on  the  rating  scale. 
However,  there  are  some  exceptions  noted  below  that  reflect  implementation  details  or  specific 
wordings  in  the  questions.  Many  of  the  items  show  a  trend  where  traditional  and  VTT  local 
students  respond  similarly  and  VTT  remote  students  give  somewhat  lower  rating.  This  pattern  is 
similar  to  that  observed  in  previous  VTT  research  (Simpson,  et  al.,  1990,  1991a,  1991b,  1992, 
1993, 1995;  Wetzel,  et  al.,  1995). 

A  one-way  analysis  of  variance  (ANOVA)  was  computed  for  each  item  to  determine  the 
statistical  significance  of  rating  differences  among  traditional,  VTT  local,  and  VTT  remote 
classrooms.  The  degrees  of  freedom  and  F  ratio  results  for  these  main  effect  tests  are  shown  in  the 
ANOVA  T-L-R  columns  of  Table  1.  If  a  main  effect  was  significant,  Tukey  HSD  tests  were  also 
computed  to  examine  which  of  the  pairwise  mean  differences  contributed  significantly  to  the 
effect.  Asterisks  in  the  last  three  HSD  columns  of  Table  1  indicate  significant  differences  for  the 
pairwise  combinations  among  traditional  (T),  VTT  local  (L),  and  VTT  remote  (R)  conditions,  i.e., 
T-L,  T-R,  L-R.  Most  of  the  mean  differences  in  Table  1  are  small  and  many  are  statistically 
significant  as  a  consequence  of  the  relatively  large  sample  sizes  (mean  differences  significant  at  the 
.05  level  are  discussed  below). 
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Question 

•  Instructors 

1.  Instructors  presented  lessons  clearly 

2.  Instructors  encouraged  class  participation 

3.  instructors  answered  student  questions  adequately 

4.  Instructors  could  be  seen  clearly 

5.  Instructors  could  be  heard  adequately 

6.  instructors  helped  students  who  needed  assistance 

•  Learning  &  Classroom  Activities 

7.  Concepts  or  theory  underlying  the  computations  were 
adequately  presented 

8.  Computational  procedures  were  adequately  presented 

9. 1  hesitated  to  ask  questions  in  order  to  clarify  concepts 
or  steps  in  the  computations 

10. 1  received  help  on  computational  problems  when  I  needed  it 

11. 1  had  access  to  the  classroom  when  class  was  not  in  session 

12. 1  had  access  to  instructors  outside  of  class 

13.  Assigned  problems  were  graded  to  provide  feedback 


Traditional 


□  VTT  Local 


V7\  VTT  Remote 


Figure  5.  Student  questionnaire  ratings  for  instructors  and  learning  activities 
(items  1-13). 


Traditional  □  VTT  Local 


VTT  Remote 


1 


Question 

•  Overall 


27.  lnstructor(s)  compare  favorably  with  the  best  Navy 
instructors  I  have  had  in  the  past 

28.  This  course  compares  favorably  with  the  best  Navy  courses 
I  have  taken  in  the  past 

29. 1  found  the  course  material  to  be  difficult 

30.  The  pace  of  the  course  was: 

[1]  Too  slow...  [3]  About  right..  [5]  Too  fast 

•  Video-Teletraining 

34.  Instructors  coordinated  activities  among  classes 
effectively 


35.  Local  and  remote  sites  participated  equally  in  the  class 


36.  The  instructor  called  on  students  at  all  of  the  sites  about 
equally 

37.  The  VTT  facilitators  assisting  the  instructor  were  helpful 
to  students  in  the  class 


WM  Traditional  |  |  VTT  Local  ^3  VTT  Remote 


Disagree  Agree 

1  2  3  4  5 


pzzzzzzzzzzr4.3 


3.3  i 

IZZZZZZZZ^is 


4.4  i 


4.4^ 


4.3 


14.5 


PZZZZZZZZZZZ4.3  i 


I  3.8 


ZZZZZZZZZ24.0 


Figure  7.  Student  questionnaire  ratings  for  course  overall  and 
videoteletraining  (items  27-30  and  34-37. 


Ratings  for  the  instructors  and  audio-visual  adequacy  were  assessed  in  items  1-6.  The  pattern 
of  results  over  these  questions  was  very  similar  and  group  differences  on  all  items  were  statistically 
significant.  The  traditional  and  VTT  local  student  ratings  were  about  a  half  of  a  rating  unit  above 
those  of  the  VTT  remote  students.  The  pattern  of  Tukey  HSD  comparisons  showed  that  the  VTT 
remote  students  gave  significantly  lower  ratings  than  either  traditional  or  VTT  local  students. 
Traditional  and  VTT  local  students  did  not  differ  significantly  from  one  another  on  these  items. 

Items  7-13  addressed  learning  and  classroom  activity  issues.  Significant  group  differences 
were  obtained  on  all  but  one  of  these  items  (item  7).  The  presentation  of  computational  procedures 
was  given  a  lower  rating  by  remote  site  students  compared  to  the  other  two  groups,  although  their 
rating  was  still  high  (item  8).  The  results  for  item  9  indicate  that  remote  site  students  were  more 
likely  to  hesitate  to  ask  questions  to  clarify  concepts  or  steps  in  the  computations.  The  remote  site 
students  differed  by  .90  and  .82  of  a  rating  unit  from  traditional  and  local  students  on  this  question. 
Similarly,  in  rating  the  help  received  on  computational  problems  when  it  was  needed  (item  10), 
remote  site  students  were  about  three  quarters  of  rating  unit  lower  relative  to  traditional  and  VTT 
local  students.  Remote  site  students  also  rated  the  feedback  on  assigned  problems  lower  than  either 
traditional  or  VTT  local  students  on  item  13. 
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Table  1 


Statistical  Comparisons  for  Student  Questionnaire  Rating  Items 


ANOVA  Mean  Differences  and 

T-L-R  or  L-R  Tukey  HSD  Comparisons 


Question 

DF 

F 

T-L 

T-R 

L-R 

Instructors 

1.  Instructors  presented  lessons  clearly 

2,237 

17.40** 

0.15 

0.58** 

0.43** 

2.  Instructors  encouraged  class  participation 

2,238 

8.47** 

0.17 

0.48** 

0.31** 

3.  Instructors  answered  student  questions 
adequately 

2,238 

17.47** 

0.16 

0.68** 

0.52** 

4.  Instructors  could  be  seen  clearly 

2,237 

15.91** 

0.07 

0.45** 

0.38** 

5.  Instructors  could  be  heard  adequately 

2,238 

18.92** 

0.10 

0.54** 

0.44** 

6.  Instructors  helped  students  who  needed 
assistance 

2,237 

16.23** 

0.20 

0.69** 

0.49** 

Learning  and  Classroom  Activities 

7.  Concepts  or  theory  underlying  the 
computations  were  adequately  presented 

2,238 

2.02 

0.10 

0.27 

0.18 

8.  Computational  procedures  were  adequately 
presented 

2,238 

9,52** 

0.12 

0.43** 

0.32** 

9. 1  hesitated  to  ask  questions  in  order  to  clarify 
concepts  or  steps  in  the  computations 

2,237 

13.02** 

-0.08 

-0.90** 

-0.82** 

10. 1  received  help  on  computational  problems 
when  I  needed  it 

2,236 

19.81** 

0.05 

0.74** 

0.69** 

11. 1  had  access  to  the  classroom  when  class  was 
not  in  session 

2,234 

19.82** 

1.15** 

0.41 

-0.74** 

12. 1  had  access  to  instructors  outside  of  class 

2,233 

39.31** 

0.70** 

1.69** 

0.99** 

13.  Assigned  problems  were  graded  to  provide 
feedback 

2,235 

22.00** 

0.21 

0.74** 

0.54** 

Training  Aids/Materials 

14.  Training  aids  were  used  effectively 

2,238 

4.21* 

-0.02 

0.22* 

0.24* 

15.  Details  of  training  aids  could  be  clearly  seen 

2,238 

5.12** 

0.02 

0.33* 

0.31* 

16.  Instructor’s  oveiheads/slides  helped  me 
locate  relevant  entries  in  the  publications 

2,237 

8.34** 

0.13 

0.48** 

0.35** 

17.  Instructor’s  visual  aids  for  strip  forms 
worked  effectively 

2,237 

10.43** 

0.04 

0.38** 

0.34** 

18.  Instructor’s  drawings  and/or  writing  during 
class  were  clear  and  understandable 

2,238 

4.85** 

0.01 

0.29* 

0.27* 

19.  The  film/videotape  was  informative 

2,237 

3.03* 

0.17 

0.38* 

0.21 

20.  The  film/videotape  was  clearly  seen  and 
heard 

2,237 

2.86 

-0.02 

0.20 

0.22 

21 .  Table  space  was  adequate  to  accommodate 
publications,  charts,  and  to  perform  plotting 

2,235 

31.30** 

1.46** 

0.79** 

-0.67** 

22.  Charts  and  publications  were  used  without 
problems 

2,237 

4.62* 

0.32* 

0.33* 

0.01 

Mean  differences  are;  T-L  =  Traditional-Local;  T-R  =  Traditional-Remote;  L-R  =  Local-Remote. 
*p<.05,**p<.0l. 

Scale  for  all  items  except  #30  is:  (1)  Strongly  Disagree,  (2)  Disagree,  (3)  Neither  Agree/Disagree,  (4)  Agree, 
(5)  Strongly  Agree. 
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Table  1  (Continued) 


ANOVA  Mean  Differences  and 

T-L-R  or  L-R  Tukey  HSD  Comparisons 

_  Qp  p  P  ^  j. 


Interaction/Participation 

23.  Interaction  between  instructors  and  students 
was  sufficient  to  support  learning  objectives 

2,236 

19.03** 

0.27* 

0.70** 

0.42** 

24.  Class  participation  was  sufficient  to  support 
learning  objectives 

2,236 

749** 

0.12 

0.41** 

0.30** 

25.  Students  aided  one  another  in  performing  the 
computations 

2,236 

1.92 

0.07 

-0.18 

-0.25 

26.  Instructor  gave  equal  attention  to  aU  students 

2,236 

8.05** 

0.22 

0.52** 

0.30* 

Overall 

27.  Instructor(s)  compare  favorably  with  the  best 
Navy  instructors  I  have  had  in  the  past 

2,237 

15.56** 

0.31 

0.83** 

0.52** 

28.  This  course  compares  favorably  with  the 
best  Navy  courses  I  have  taken  in  the  past 

2,237 

5.24** 

0.29 

0.51** 

0.22 

29. 1  found  the  course  material  to  be  difficult 

2,238 

2.28 

-0.03 

-0.38 

-0.35 

30.  The  pace  of  the  course  was:  (1)  too  slow, 

(2)  somewhat  slow,  (3)  about  right, 

(4)  somewhat  fast,  or  (5)  too  fast 

2,237 

0.96 

-0.11 

-0.18 

-0.07 

Video-Teletraining 

34.  Instructors  coordinated  activities  among 
classes  effectively 

1,185 

4.06* 

0.17* 

35.  Local  and  remote  sites  participated  equally 
in  the  class 

1,185 

1.76 

0.13 

36.  The  instructor  called  on  students  at  all  of  the 
sites  about  equally 

1,183 

5.51* 

0.21* 

37.  The  VTT  facilitators  assisting  the  instructor 
were  helpful  to  students  in  the  class 

fVlPQTl  '1'  T  '  I .4 : 4-1  1  T _ 1-  "Tri 

1,176 

2.46 

— 

— 

-0.22 

Mean  differences  are:  T-L  =  Traditional-Lxxal;  T-R  =  Traditional-Remote;  L-R  =  Local-Remote 
*;7<.05,**p<.01. 


Scale  for  all  items  except  #30  is:  (1)  Strongly  Disagree,  (2)  Disagree,  (3)  Neither  Agree/Disagree,  (4)  A«ree 
(5)  Strongly  Agree.  ’ 
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Items  11  and  12  appear  to  reflect  a  mix  of  implementation  details  particular  to  the  VTT  classes. 
For  item  11,  access  to  the  classroom  when  the  class  was  not  in  session  was  rated  lower  by  VTT 
local  students  than  either  traditional  or  VTT  remote  students.  This  is  probably  related  to  the  hours 
at  which  the  VTT  local  classroom  was  secured,  the  instructors  returning  to  their  traditional  work 
spaces  in  another  building  near  the  end  of  the  day,  and  the  fact  that  the  instructors'  office  was 
adjacent  to  the  classroom  for  traditional  classes.  On  item  12,  these  circumstances  may  also  account 
for  VTT  local  students  having  given  lower  ratings  than  traditional  students  on  their  access  to  the 
instructor  outside  of  class.  However,  access  to  instructors  outside  of  class  is  most  severely  reduced 
for  VTT  remote  students  relative  to  either  VTT  local  (.99  lower)  or  traditional  students  (1.69 
lower).  The  2.6  average  rating  of  VTT  remote  students  is  the  lowest  rating  obtained  relative  to  the 
other  groups  over  all  the  items  in  the  questionnaire. 

Items  14-22  addressed  the  training  aids  and  materials  used  in  the  course.  With  the  exception  of 
one  item  (21),  most  of  these  items  revealed  small  differences  among  the  treatment  groups  that  did 
not  exceed  one  half  of  a  rating  unit 

For  items  14  through  20,  the  traditional  and  VTT  local  students  responded  virtually  the  same 
and  the  VTT  remote  students  generally  gave  ratings  that  were  about  one  third  to  one  fifth  of  a  rating 
point  lower  (a  significant  overall  effect  was  found  on  each  of  the  items  but  item  20).  VTT  remote 
students  gave  slightly  lower  ratings  on  these  items  concerned  with  the  effectiveness  or  visibility  of 
the  training  aids  shown  over  the  system  (i.e.,  use,  visibility  of  details,  locating  entries  in 
publications,  strip  form  visual  aids,  instructor  drawings  and  writings,  and  the  videotaped 
instructional  film  shown). 

All  three  treatment  groups  differed  significantly  from  one  another  on  item  21  where  the 
adequacy  of  table  space  for  publications,  charts,  and  plotting  was  rated.  The  traditional  classroom 
for  this  course  has  large  plotting  tables,  whereas  many  VTT  classrooms  are  equipped  with  24"  x 
60"  tables  for  a  pair  of  students  which  are  suitable  for  typical  lecture-based  courses.  Accordingly, 
the  ratings  by  traditional  students  are  high,  but  VTT  local  students  give  the  lowest  ratings,  with 
VTT  remote  students  giving  ratings  intermediate  between  these  two  other  groups.  This  pattern  of 
response  reflects  the  fact  that  during  the  period  of  this  study  the  remote  sites  contained  larger  tables 
and  fewer  students,  whereas  the  VTT  local  site  had  many  more  students  sharing  physically  smaller 
tables  so  that  the  available  space  per  student  was  less.  In  rating  how  well  charts  and  publications 
could  be  used  in  item  22,  both  local  and  remote  students  responded  similarly  and  gave  ratings  about 
a  third  of  a  rating  unit  below  the  traditional  students. 

Items  23-26  addressed  interaction  and  participation  in  the  course.  A  similar  significant  pattern 
of  response  was  found  on  items  23,  24,  and  26.  Traditional  students  rated  interaction  the  highest, 
followed  closely  by  VTT  local  students,  and  VTT  remote  students  gave  the  lowest  ratings.  These 
ratings  indicate  that  instructor-student  interaction,  class  participation,  and  receiving  equal  attention 
from  the  instructor  were  perceived  to  be  less  by  VTT  remote  students  (the  group  differences  ranged 
from  a  third  to  three  quarters  of  a  rating  point).  On  item  25,  the  slight  tendency  of  remote  site 
students  to  agree  that  students  aided  one  another  in  performing  computations  was  not  found 
significant.  However,  as  will  be  seen  later  in  discussing  item  38,  remote  site  students  were  more 
likely  to  cite  other  students  as  a  source  of  assistance  outside  of  class. 

Overall  ratings  of  the  instruction  were  assessed  in  items  27-30.  Significant  group  differences 
were  found  in  rating  the  instructors  (item  27)  and  the  course  (item  28)  relative  to  prior  instructors 
or  courses.  VTT  remote  students  gave  the  lowest  ratings,  followed  by  VTT  local  students,  but  these 
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were  still  in  the  “agree”  region  of  the  rating  scale.  VTT  remote  students  gave  significantly  lower 
ratings  for  instructors  than  traditional  and  V 11  local  students,  and  gave  significantly  lower  ratings 
of  the  course  than  traditional  students. 

Items  29  and  30  were  included  in  the  questionnaire  to  assess  an  expectation  that  remote  site 
students  might  find  the  course  material  more  difficult  or  be  more  sensitive  to  the  pace  of  the  course. 
Although  the  pattern  of  responses  are  slightly  in  this  direction,  there  were  no  significant  group 
differences  to  bear  out  the  expectation.  Students  generally  judged  both  the  difficulty  and  the  pace 
of  the  course  to  be  just  about  right,  with  ratings  on  these  items  falling  close  to  the  midpoint  of  the 
scale. 

Rating  items  34-37  were  completed  only  by  VTT  local  and  remote  students  because  these  items 
pertained  only  to  VTT  issues.  Group  differences  on  these  items  were  negligible,  with  the  largest 
differences  being  only  about  a  fifth  of  a  rating  point  lower  for  remote  students  than  local  students. 
Significant  differences  were  shown  on  items  34  and  36,  where  remote  students  gave  slightly  lower 
ratings  for  instructors  coordinating  activities  among  classes  and  calling  on  all  students  equally. 

Multiple  Choice  Items 


Items  38-41  were  given  in  a  multiple  choice  format  to  assess  student  opportunities  to  interact 
with  the  instructor,  assistance  received,  and  preferences  for  an  instructional  method.  Percentages 
of  response  to  each  choice  were  calculated  for  local  and  remote  classrooms,  and  Chi  Square  tests 
were  used  to  compare  response  distributions  by  classroom. 

Opportunities  to  interact  with  instructor.  Figure  8  shows  student  responses  to  question  40, 
which  asked  How  did  the  VTT  method  of  instruction  affect  your  opportunities  to  interact  with  the 
instructor?  Nearly  nine  tenths  of  the  VTT  local  students  responded  that  their  opportunities  were 
unaffected.  The  majority  of  VTT  remote  students  were  divided  in  their  opinion  between  there 
being  no  effect  (45.9%)  or  fewer  ’  opportunities  (42.6%).  Comparing  the  two  groups,  fewer 
opportunities  to  interact  with  the  instructor  were  reported  by  remote  students  (44.6%)  than  by  local 
students  (7.4%).  A  Chi  Square  test  showed  that  the  distribution  of  responses  were  significantly 
different  (xT2)  =  40.78,  p  <  .001). 


Opportunities  to  Interact  with  instructor 


Classroom 


Figure  8.  Students  responses  on  how  VTT 
affected  opportunities  to  interact 
with  the  instructor  (item  40). 
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Assistance.  Question  38  asked  “Who  most  frequently  provided  assistance  to  you  outside  of 
regular  class  hours”  and  offered  choices  in  terms  of  “Instructor,”  “Facilitator,”  “Other  students,” 
and  “Other  (please  explain).”  Results  to  this  question  are  shown  in  Figure  9.  A  Chi  Square  test 
showed  that  the  distribution  of  responses  among  the  treatment  groups  was  significantly  different 
(%^(6)  =  76.05,  p  <  .001). 

Source  of  Student  Assistance  Outside  of  Class 


Instructor  Other  Students  Facilitator  Other  or 

No  One 

Figure  9.  Student  responses  on  who  most  frequently  provided  assistance 
outside  of  regular  class  hours  (item  38). 

The  main  result  of  interest  in  the  responses  was  that  VTT  remote  students  cited  other  students 
as  the  main  source  of  assistance  outside  of  class  (59.3%)  much  more  often  than  did  the  other 
groups.  The  instructor  was  least  often  the  source  of  assistance  for  VTT  remote  students,  whereas 
the  instructor  was  most  often  cited  as  a  source  by  traditional  and  VTT  local  students.  Remote 
students  clearly  cited  other  students  as  a  source  of  assistance  in  this  question,  but  as  discussed  for 
item  25  they  were  only  slightly  more  likely  to  agree  that  students  aided  one  another  in  performing 
computations.  However,  item  38  included  the  phrasing  “outside  of  regular  class  hours”  so  that  this 
question  may  have  more  explicitly  referred  to  nonlecture  laboratory  periods  at  the  end  of  the  day 
when  the  instructor  was  not  directing  activities. 

Local  students  were  a  little  less  likely  than  traditional  students  to  cite  the  instructor  as  a  source 
of  assistance  and  a  little  more  likely  to  cite  other  students.  This  result  is  probably  related  to  the 
explanation  offered  for  the  responses  to  items  11  and  12.  It  was  observed  that  the  instructor  office 
was  adjacent  to  the  classroom  for  traditional  classes,  whereas  during  VTT  classes  the  instructors 
returned  to  their  traditional  work  spaces  near  the  end  of  the  day. 


The  Other  category  was  chosen  by  one  fifth  to  one  third  of  the  students  over  the  three  groups. 
Of  those  choosing  this  alternative,  about  half  (52%)  indicated  no  help  was  needed,  38%  left  the 
blank  empty,  and  9%  cited  either  a  combination  of  response  categories  or  that  they  received  help 
from  someone  at  their  command. 

Preference  for  Instructional  Method.  Figure  10  reports  VTT  student  preferences  for  a 
method  of  instruction  and  their  willingness  to  participate  in  another  VTT  course  (questions  39  and 
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Preference  for  a  Method  of  Instruction 


Willingness  to  Participate 
in  Another  VTT  Course 


Traditional 


VTT  Local 
Classroom 


VTT  Remote 


VTT  Local  VTT  Remote 
Classroom 


Figure  10.  Student  preferences  for  a  method  of  instruction  and  willingness 
to  participate  in  another  VTT  class. 


The  left  hand  panel  of  Figure  10  shows  the  results  for  question  39,  which  asked  “Which  method 
of  instruction  would  you  prefer  for  this  course?”  The  responses  offered  were:  Video  Tele-Training 
(VTT),  Traditional  method  (non- VTT  ),  and  Either  method.  Traditional  students  were  given  a  short 
description  of  VTT  along  with  this  question.  A  Chi  Square  test  showed  that  the  distribution  of 
responses  to  these  categories  differed  significantly  among  the  treatment  groups  (x^(4)  =  50.46,  p  < 
.001). 

Traditional  students  were  very  strongly  in  favor  of  the  traditional  form  of  instruction  (70.8%), 
followed  by  either  method  (22.9%)  and  then  the  VTT  method  of  instruction  (6.3%).  By  contrast, 
the  preference  for  the  majority  of  VTT  local  students  was  for  “either”  method  (62.9%),  followed 
by  the  traditional  (25%)  and  then  the  VTT  (12.1%)  methods  of  instruction.  Compared  to  the  these 
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two  student  groups,  the  VTT  remote  students  were  more  divided  in  their  preferences.  Among  VTT 
remote  students,  about  a  tenth  more  chose  “either”  method  (44.3%)  than  chose  the  VTT  method 
(32.8%).  The  remaining  quarter  of  the  remote  students  chose  the  traditional  method  (23%)  about 
as  often  as  did  the  local  students.  VTT  remote  students  showed  the  highest  percentage  of  choice 
for  the  VTT  method  among  the  three  treatment  groups  (followed  by  VTT  local  and  then 
traditional). 

The  finding  that  VTT  remote  students  were  more  favorable  toward  VTT  is  notable  because 
these  students  were  at  a  distance  from  the  instructor  and  had  experienced  the  full  impact  of  the 
instructional  method.  Student  acceptance  of  VTT  increases  with  greater  direct  experience  with  the 
impact  of  the  VTT  method  when  the  group  responses  are  ordered  from  traditional  to  VTT  local  to 
VTT  remote.  The  results  support  previous  observations  with  instructional  television  that  students 
may  initially  have  negative  perceptions,  but  these  lessen  with  experience  with  the  medium  and 
become  neutral  or  even  positive  (cf.  Wet2el,  et  al.,  1993, 1994). 

The  right  hand  panel  of  Figure  10  shows  the  results  for  question  41.  Students  were  asked  to 
respond  “Yes”  or  “No”  to  the  question  “If  you  had  a  choice,  would  you  take  another  VTT  course?” 
An  overwhelming  nine  tenths  of  the  students  at  either  local  or  remote  sites  indicated  that  they 
would  take  another  VTT  course.  A  Chi  Square  test  showed  no  significant  difference  in  the 
distribution  of  Yes  and  No  responses  among  the  two  treatment  groups  (x^(l)  =  0.005,  p>  .05). 

Students  were  asked  to  “Please  explain”  their  yes  or  no  response  in  the  second  part  of  question 
41.  About  half  of  the  students  did  not  provide  a  comment  (50%  local;  42.6%  remote).  About  a  third 
of  the  students  offered  positive  comments,  such  as  savings  in  travel  costs  or  the  ability  to  teach 
more  students  (33.6%  local;  36.1%  remote).  A  tenth  of  the  comments  were  negative,  such  as 
students  being  distant  from  the  instructor  and  their  ability  to  get  assistance,  or  technical  problems 
with  VTT  (11.7%  local;  9.8%  remote).  The  remaining  students  wrote  a  combination  of  both 
positive  and  negative  comments  (4.7%  local;  11.5%  remote).  Thus,  the  most  frequent  comment  by 
those  who  responded  was  positive  and  the  distribution  of  responses  among  the  two  treatment 
groups  was  very  similar  and  did  not  differ  significantly  (x^(3)  =  3.44,  p  >  .05). 

Open*Ended  Questions 

Questions  42, 43,  and  44  asked  students  what  they  liked  most  about  the  course,  what  they  liked 
least,  and  to  provide  any  suggestions  to  improve  the  course,  respectively.  The  written  responses  to 
each  open-ended  question  were  clustered  into  several  categories  and  percentages  within  each 
treatment  group  were  calculated.  The  results  will  be  discussed  briefly  here  because  the  responses 
to  these  questions  did  not  reveal  substantial  differences  among  the  treatment  groups  (summary 
tables  are  given  in  Appendix  E). 

Comparing  the  three  questions  over  all  treatment  groups,  students  were  more  likely  to  respond 
about  what  they  liked  most  (82%),  than  what  they  liked  least  (64%),  or  to  suggest  improvements 
(51%).  Chi  Square  tests  showed  that  the  likelihood  of  responding  with  a  comment  did  not  differ 
significantly  among  the  three  treatment  groups  on  each  of  the  three  questions  (x^(2)  =  1.8, 3.1, 3.2, 
p  >  .05,  for  items  42, 43,  and  44  respectively). 


27 


Student  comments  on  what  they  liked  most  about  the  course  (item  42)  most  frequently  cited 
the  subject  matter  as  a  valuable  skill  relevant  to  their  jobs  (28%  over  all  groups).  The  second  most 
frequent  comment  involved  three  related  categories  pertaining  to  the  value  of  the  exercises, 
practice,  and  plotting;  performing  calculations;  or  the  value  of  these  instructional  strategies  for 
practical  application  (20%  over  all  groups).  The  next  most  frequent  comment  categories  involved 
positive  comments  about  the  instructor,  the  appropriate  pace  of  the  course,  and  positive  aspects  of 
using  the  VTT  system.  Together,  these  six  categories  comprised  about  70%  of  the  comments  given 
on  what  was  liked  most  about  the  course.  Relative  to  the  other  groups,  positive  comments  by 
remote  students  were  more  frequent  with  regard  to  VTT  and  a  little  less  frequent  with  regard  to 
instructional  strategies,  exercises,  homework,  and  instructors. 

Student  comments  on  what  was  liked  least  about  the  course  (item  43)  most  frequently  cited  the 
amount  or  repetitive  nature  of  the  calculations  performed  for  homework  (18.5%  over  all  groups). 
The  second  most  frequent  category  cited  the  pace  or  length  of  the  course  (12.5%  overall).  The 
remaining  responses  were  distributed  among  several  categories,  with  a  slight  tendency  for  remote 
students  to  comment  on  the  instructor  or  for  local  students  to  comment  on  the  facilities.  Student 
comments  suggesting  improvements  for  the  course  (item  44)  fell  in  diverse  categories  and  tended 
to  parallel  those  on  what  was  liked  least  about  the  course.  About  half  of  the  smdents  provided 
comments  and  the  most  frequent  cluster  referred  to  details  about  the  instructional  materials, 
strategies,  and  exercises  (such  as  homework,  forms,  and  more  hands-on  work  with  plotting  or  the 
sextant).  The  next  most  frequent  categories  concerned  facilities  issues  such  as  table  space  for 
plotting,  the  pace  of  the  course,  and  knowledge  of  celestial  navigation  by  facilitators. 

Summary  of  Related  Questionnaire  Topics 

Student  responses  on  the  rating  items  were  consistently  higher  for  VTT  local  and  traditional 
students  than  for  VTT  remote  students.  About  two  thirds  of  the  items  involved  a  statistically 
significant  less  positive  rating  for  remote  students  compared  to  local  or  traditional  students  (the 
latter  two  groups  differed  significantly  in  only  about  a  tenth  of  the  items).  However,  the  magnitude 
of  the  differences  between  the  treatment  groups  was  generally  not  large.  Additionally,  the  ratings 
for  all  groups  were  generally  high  and  in  a  positive  direction  (about  one  unit  away  from  the  most 
positive  rating  on  a  five  point  scale).  This  observed  pattern  and  magnitude  of  differences  is  typical 
of  that  seen  in  previous  VTT  research  (Simpson,  et  al.,  1990,  1991a,  1991b,  1992,  1993  1995- 
Wetzel,  et  al.,  1995). 

A  number  of  the  questionnaire  items  pertained  to  similar  topics  even  though  they  were  not 
grouped  together  in  the  questionnaire  as  administered  to  students.  The  following  paragraphs 
summarize  the  general  pattern  of  findings  for  these  topic  areas.  The  magnitude  of  the  difference 
between  the  groups  differed  somewhat  over  these  different  topic  areas. 

For  those  questions  relating  to  participation  issues,  ratings  by  remote  students  were  about  one 
quarter  of  one  rating  unit  lower  than  local  students  and  one  half  a  unit  lower  than  traditional 
students  (items  2,  23,  24,  26,  34,  35,  36).  Responses  were  very  similar  to  questions  relating  to 
audio-visual  factors  that  had  to  do  with  hearing  and  seeing  the  instruction  (items  4,  5,  15,  16,  17, 
18,  20).  Remote  site  students  gave  ratings  only  about  one  third  of  a  unit  lower  than  local  or 
traditional  students.  The  responses  suggest  that  the  various  small  print  publications  and  strip  forms 
were  successfully  adapted  for  delivery  by  VTT  and  were  not  a  problem.  For  questions  on  instructor 
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presentations  of  instructional  content,  remote  students  gave  ratings  about  a  third  of  one  unit  lower 
than  local  students  and  not  quite  a  half  a  unit  lower  than  traditional  students  (items  1, 7, 8, 14, 27). 

For  those  questions  related  to  getting  help,  assistance,  feedback,  or  attention,  remote  students 
gave  ratings  about  a  half  a  unit  lower  than  local  students  and  about  two  thirds  a  unit  lower  than 
traditional  students  (items  3,  6,  10, 13,  26).  However,  ratings  of  the  difficulty  or  faster  pace  of  the 
course  were  neutral  and  only  a  third  to  a  tenth  of  a  unit  greater  for  remote  students  than  the  local 
and  traditional  students  (items  29,  30).  Remote  students  were  noticeably  more  likely  than  local 
students  to  indicate  that  they  had  less  access  to  the  instructor  (item  12)  by  a  full  rating  unit  and  were 
about  30%  more  likely  to  say  they  had  fewer  opportunities  to  interact  with  the  instructor  (item  40). 
Remote  students  were  also  about  eight  tenths  of  a  unit  more  likely  to  indicate  a  hesitancy  to  ask 
questions  (item  9). 

Facilities  related  questions  reflected  conditions  during  the  study  period,  such  as  access  to  the 
room,  that  table  space  for  materials  was  more  confined  at  some  facilities,  but  that  the  use  of  charts 
was  not  particularly  a  problem  (items  11,  21,  22).  Providing  larger  rooms  and  tables  to  better 
accommodate  this  and  other  courses  with  hands-on  laboratories  should  be  included  in  future  VTT 
site  planning. 

Taken  together,  the  largest  difference  between  groups  reflected  a'  reduced  access  to  the 
instructor  and  assistance  related  issues  for  students  at  a  distance,  as  well  as  some  site  specific 
facilities  issues.  However,  remote  students  were  willing  to  take  another  VTT  course  and  their 
preferences  for  a  method  of  instruction  for  this  course  were  not  less  favorable  than  local  students. 

Instructor-Student  Interaction  and  Class  Participation 

The  tally  of  interactions  in  the  course  was  recorded  to  assess  the  relative  level  of  instructor- 
student  interaction  among  local  and  remote  sites  and  to  assess  whether  the  level  of  participation  by 
remote  student  was  adequate. 

Interaction  tallies  were  recorded  during  the  first  three  days  of  class  during  six  class  convenings 
(one  local  and  one  remote  classroom  were  always  present  and  a  second  remote  site  was  present 
during  three  of  the  convenings).  There  were  twice  as  many  local  site  students  (68.7%)  as  remote 
site  students  (31.3%)  (101  local  and  46  remote  students  for  a  total  of  147  total  students,  which 
attrition  reduced  to  100  local  and  45  remote  students  by  day  3). 

Categories  and  Measures 

Interaction  Tally  Categories.  An  observer  recorded  interactions  in  the  seven  tally  categories 
that  were  defined  earlier  in  the  method  section.  Instructor  questions  (categories  1, 2,  and  3)  which 
were  answered  by  a  student  from  either  the  local  or  remote  site  were  questions  that  were  either  (1) 
open  to  any  site,  (2)  named  a  site,  or  (3)  identified  a  specific  student.  Student  questions  (categories 
4  and  5)  were  asked  by  either  a  local  or  remote  site  student  and  were  (4)  student  initiated  questions 
or  (5)  conversations  and  exchanges  extending  beyond  a  single  question.  Rater  agreement  was 
found  to  be  generally  high  for  these  tally  categories  when  two  raters  observed  the  same  class 
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periods.  Three  other  miscellaneous  categories  (6,  7,  8)  were  also  recorded:  (6)  no  answer  to  an 
instructor  question,  (7)  no  answer  to  a  generic  instructor  query  for  “any  questions?”,  and  (8) 
instructor  reminders  to  local  site  students  to  use  the  push-to-talk  microphones.  Interactions 
recorded  during  the  six  class  convenings  resulted  in  a  total  3,428  interactions  for  all  eight  of  these 
categories  and  2,847  for  the  first  five  categories. 

Interaction  Measures.  R  The  interaction  counts  were  converted  to  two  measures  that  adjusted 
for  variations  in  the  length  of  time  for  class  periods  and  the  unequal  numbers  of  students  between 
local  and  remote  sites.  These  two  measures  were  (a)  interactions  per  hour,  and  (b)  interactions  per 
hour  per  student  Tables  in  Appendix  F  illustrate  the  calculation  of  these  measures  and  present 
various  summary  tables  supporting  the  descriptions  of  the  interaction  patterns  that  are  given  below. 

The  interactions  per  hour  measure  compensates  for  recording  periods  that  were  not  always 
^ual  for  various  reasons  (e.g.,  over  different  class  convenings,  days  of  the  week,  and  class  periods 
in  the  morning  and  afternoon).  This  measure  reflects  the  rate  of  interaction  for  an  average  class  in 
an  hour.  This  measure  gives  the  actual  level  of  interaction  in  the  classes  observed  and  does  reflect 
differences  in  the  number  of  students  between  local  and  remote  sites. 

The  interactions  per  hour  per  student  measure  allows  local  and  remote  sites  with  unequal 
numbers  of  students  to  be  compared  in  terms  of  the  level  of  interaction  relative  to  how  many 
students  were  available  to  contribute  interactions.  This  measure  adjusts  for  both  unequal  time 
periods  and  unequal  numbers  of  students.  The  measures  can  be  interpreted  to  reflect  the  rate  of 
interaction  appropriate  to  an  average  class  relative  to  the  number  of  students  contributing 
interactions. 

Instructor  and  Student  Questions 

The  upper  (a)  and  lower  (b)  panels  of  Figure  1 1  show  the  results  for  VTT  local  and  remote  sites 
m  terms  of  each  of  the  two  interaction  measures  respectively.  Each  panel  shows  the  individual 
instructor  and  student  question  categories  (1  to  5),  and  the  result  of  combining  the  three  instructor 
(1  to  3)  and  two  student  (4  and  5)  categories. 

The  figure  shows  that  the  Celestial  Navigation  classes  were  heavily  “instructor  driven”  because 
most  interaction  resulted  from  instructor  questions  rather  than  from  student  questions.  Relative  to 
die  total  of  all  instructor  and  student  question  categories  (1  to  5),  78%  of  the  interactions  were 
instructor  questions  (categories  1  to  3)  and  22%  were  student  questions  or  conversation  exchanges 
(categories  4  and  5). 

The  figure  also  shows  that  instructors  used  a  mix  of  different  types  of  questions  that  were  open 
to  any  site,  directed  at  a  site,  or  directed  at  a  specific  student.  The  mixture  of  instructor  questions 
would  be  judged  beneficial  because  the  instructor  can  assess  knowledge  levels  in  both  those 
students  likely  to  respond  when  they  know  an  answer  and  those  individuals  who  are  less  likely  to 


The  Pearson  correlation  between  two  raters  observing  the  same  sessions  during  three  halfday  periods  was  0.975 
for  30  pairs  of  observations  (perfect  correspondence  would  be  1 .0).  Observer  agreement  was  high  within  the  instructor 
question  and  student  initiated  questions  categories  (ranging  from  0.939  to  0.993),  but  was  somewhat  lower  in  judging 
student  conversations  and  exchanges  (0.667).  Additional  details  on  this  data  are  given  in  Table  F-2  of  Appendix  F. 
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participate.  Relative  to  the  total  of  all  instructor  and  student  question  categories  (1  to  5),  32%  were 
instructor  questions  open  to  any  site,  20%  named  a  site,  and  26%  were  directed  toward  a  specific 
student  Student  initiated  questions  were  16%  of  this  total,  and  student  conversations  or  exchanges 
were  6%. 


Instructor  Questions: 

(1)  Open  to  Any  Site 

(2)  Names  a  Site 

(3)  Specific  Student 
Student  Questions: 

(4)  Student  Initiated 

(5)  Conversations 
Combined: 

(1*3)  All  Instructor  Question 
(4*5)  All  Student  Questions 


■5 - TO - T5 - TO" 

Interactions  Per  Hour 


Instructor  Questions: 

(1)  Open  to  Any  Site 

(2)  Names  a  Site 

(3)  Specific  Student 
Student  Questions: 

(4)  Student  Initiated 

(5)  Conversations 
Combined: 

(1-3)  All  Instructor  Question 
(4-5)  All  Student  Questions 


VTT  Local  t/A  VTT  Remote 


Interactions  Per  Hour  Per  Student 


Figure  11.  Interaction  tally  results  for  instructor  and  student  questions 
in  terms  of  (a)  interactions  per  hour,  and  (b)  interactions  per 
hour  per  student 
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Figure  11a  shows  the  interactions  on  a  per  hour  basis  and  reflects  the  actual  overall  level  of 
interaction  in  the  course  where  local  students  outnumbered  remote  students.  There  is  a  higher  level 
of  local  than  remote  site  interaction  for  instructor  questions  open  to  any  site  and  those  directed 
toward  a  specific  student.  Student  initiated  questions  are  also  greater  at  the  local  site.  However, 
instructor  questions  that  named  a  local  or  remote  site  were  much  higher  for  the  remote  site.  The 
last  bars  in  Figure  11a  show  that  the  overall  level  of  instructor  questions  to  local  and  remote 
students  is  nearly  the  same  when  all  three  instructor  question  categories  are  combined  (1  to  3).  This 
is  primarily  due  to  the  large  number  of  instructor  questions  that  named  a  site.  When  the  two  student 
question  categories  are  combined  (4  and  5),  the  local  students  are  still  more  likely  to  initiate 
questions. 

Figure  lib  shows  the  interactions  on  a  per  hour  per  student  basis  that  weights  the  rate  of 
interaction  in  terms  of  the  number  of  local  and  remote  students.  By  comparing  the  top  and  bottom 
portions  of  Figure  1 1  (a  and  b),  it  is  apparent  that  there  is  a  generally  lower  level  of  interaction  for 
remote  than  local  sites  in  (a),  but  a  higher  level  of  remote  than  local  site  interaction  in  (b).  All  five 
instructor  and  student  question  categories  show  at  least  a  slight  advantage  for  remote  site  students. 
Thus,  relative  to  their  fewer  numbers,  remote  site  student  interactions  are  not  at  a  lower  level  than 
that  of  the  local  site.  When  all  three  instructor  categories  (1  to  3)  are  combined  the  remote  site 
students  show  a  level  of  interaction  about  twice  that  of  the  local  students.  With  all  student  questions 
combined  (4  and  5)  the  remote  students  are  only  slightly  more  likely  than  remote  students  to  ask 
questions. 

The  results  with  this  course  can  be  compared  to  those  from  a  previous  study  with  Navy 
leadership  courses  where  only  student  initiated  questions  and  comments  were  recorded  (Wetzel,  et 
al.,  1995).  For  the  leadership  courses,  traditional  and  local  students  had  about  1.5  interactions  per 
student  per  hour  compared  to  1.0  for  remote  classes.  In  the  present  study,  the  level  for  local  students 
was  0.42  and  was  0.62  for  remote  students.  Thus,  there  was  a  lower  level  of  student  initiated 
questions  and  comments  for  the  Celestial  Navigation  course  than  in  leadership  courses  which 
characteristically  involve  a  more  interactive  content. 

Changes  Over  Days.  The  level  of  interaction  generally  increased  over  the  three  days,  but 
these  changes  varied  somewhat  by  category  (tables  in  Appendix  F  summarize  these  data).  Perhaps 
the  most  interesting  change  was  that  students  were  about  twice  as  likely  to  ask  questions  after  the 
first  day  of  class.  There  were  7.5,  14.4,  and  13.6  questions  per  hour  for  days  one  to  three 
respectively  (for  student  question  categories  4  and  5  combined  over  both  local  and  remote  sites). 
Instructor  questions  increased  mostly  on  the  third  day,  possibly  reflecting  topics  on  that  day  (38.9, 
39. 1,  and  52.4  questions  per  hour  for  the  three  days  with  categories  1  to  3  combined  over  local  and 
remote  sites). 

This  observed  increase  in  the  level  of  interaction  over  days  was  confirmed  by  an  analysis  of 
variance  (ANOVA).  A  two-factor  ANOVA  was  computed  using  the  two  treatment  groups  as  a 
between-subjects  factor  (local  versus  remote)  and  the  three  days  were  treated  as  a  within-subjects 
factor.  This  analysis  is  exploratory  since  there  were  only  six  class  convenings  to  be  used  as  subjects 
in  the  analysis,  which  is  somewhat  small  for  significance  testing.  The  interactions  per  hour  measure 
was  calculated  individually  for  each  of  the  six  classes  for  each  day  and  treatment  group.  These 
individual  ratios  were  computed  for  an  ANOVA  of  the  combined  instructor  questions  (sum  of 


categories  1  to  3)  and  then  for  an  ANOVA  of  the  combined  student  questions  (sum  of  categories  4 
and  5). 

The  ANOVAs  indicated  only  a  significant  increase  in  interactions  per  hour  over  the  three  days 
of  class,  as  described  earlier  in  this  section.  For  the  instructor  questions  (categories  1  to  3 
combined),  the  effect  of  days  was  significant  (F(2,20)  =  23.74,  p  <  .01),  but  neither  the  effect  of 
treatment  groups  (F(l,10)  =  1.28,/?  >  .05),  or  the  interaction  with  days  were  significant  (F(2,20)  = 
0.27,  p  >  .05).  For  the  student  questions  (categories  4  and  5  combined),  the  effect  of  days  was 
significant  (F(2,20)  =  8.07,/?  <  .01),  but  neither  the  effect  of  treatment  groups  (F(l,10)  =  1.48,/?  > 
.05),  or  the  interaction  with  days  were  significant  (F(2,20)  =  1.88,/?  >  .05). 

Had  these  analyses  alternatively  been  conducted  using  the  interactions  per  hour  per  student 
measure,  the  same  pattern  of  findings  would  be  observed  except  for  a  significantly  greater  level  of 
instructor  questions  for  remote  than  local  students  (F(l,  10)=  14.62,  p<.01).  These  analyses  support 
the  patterns  shown  in  the  grand  totals  of  Figure  11:  there  were  small  treatment  group  differences 
with  the  per  hour  measure,  and  a  larger  group  difference  for  instructor  questions  using  the  per  hour 
per  student  measure.  That  is,  the  first  measure  generally  shows  small  differences  among  groups 
despite  there  being  half  as  many  remote  students,  and  the  second  measure  shows  that  the  instructor 
tended  to  question  remote  students  more  often  relative  to  their  fewer  numbers. 

Unanswered  Questions.  There  was  a  generally  low  level  of  instructor  questions  that  were  not 
answered  by  students.  These  unanswered  questions  were  either  to  a  specific  instructor  question 
(category  6),  or  to  the  instructor’s  generic  queries  for  “any  questions?”  that  were  used  as  a  pause 
before  proceeding  with  the  instruction  (category  7).  Relative  to  all  unanswered  and  answered 
instructor  questions  combined  (the  sum  of  categories  1,  2,  3,  6,  and  7),  4.9%  were  unanswered 
specific  instructor  questions  and  12.4%  were  generic  queries  for  “any  questions?” 

Students  were  apparently  more  likely  to  provide  some  response  to  specific  instructor  question 
later  in  the  week  (category  6).  These  unanswered  questions  declined  by  about  two  thirds  after  the 
first  day  of  class  (4.5, 1.3  and  1.5  events  per  hour  for  days  one  to  three  respectively). 

In  a  pattern  similar  to  that  with  the  other  instructor  questions  (category  1  to  3),  the  generic 
query  for  “Any  Questions?”  increased  later  in  the  week  (5.4, 5.9,  and  8.1  events  per  hour  for  days 
one  to  three). 

Reminders  to  Use  Microphones.  Whereas  VTT  remote  students  must  use  microphones  to  talk 
to  the  local  site,  VTT  local  students  in  the  same  room  with  the  instructor  can  speak  directly  to  the 
instructor  without  using  the  microphones.  Although  VTT  local  students  are  told  to  use  the  push- 
to-talk  microphones  when  asking  questions  early  on  the  first  day  of  class,  they  occasionally  need 
to  be  reminded  so  that  their  questions  can  be  heard  by  remote  site  students  (interaction  category  8). 
Figure  12  shows  the  number  of  instructor  reminders  per  hour  that  were  given  to  the  VTT  local 
students  over  the  morning  and  afternoon  for  the  first  three  days  of  class.  The  number  of  reminders 
decreases  regularly  over  days  from  three  reminders  per  hour  on  the  first  morning  to  half  that  on  the 
third  afternoon.  The  level  for  mornings  and  afternoons  is  similar  on  the  last  two  days,  but  on  the 
first  day  a  decrease  is  shown  within  the  day.  The  results  suggest  that  VTT  local  students  learn  to 
use  their  microphones  over  days  because  instructor  admonishments  to  do  so  decline. 
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Figure  12.  Instructor  reminders  per  hour  to  local  site  students  to  use 
microphones. 


Technology  Evaluation  Questionnaires 

Picture-in-Picture  Evaluation 

The  evaluation  of  picture-in-picture  (PIP)  technology  used  in  the  course  involved  presenting 
periods  of  instruction  with  PIP  enabled  or  disabled  (“Alternating  PIP”  treatment),  or  with  PIP 
continuously  enabled  throughout  the  course  (“Regular  PIP  Use”). 

Figure  13  shows  the  average  ratings  for  eight  of  the  rating  items  from  the  PIP  Questionnaire 
given  in  Appendix  B.  The  first  four  items  shown  were  rated  near  the  “[4]  Agree”  region  of  the  five 
point  scale,  and  the  last  four  items  were  rated  near  the  “[2]  Disagree”  region  of  the  scale.  The 
overall  pattern  of  ratings  between  different  questions  suggests  the  following  student  perceptions. 
The  quality  of  the  presentation  was  improved  by  using  PIP,  it  was  beneficial  to  see  both  the 
instructor  and  visuals  at  the  same  time,  and  PIP  was  used  appropriately.  The  PIP  image  was  clear 
and  there  were  not  usually  technical  problems.  Students  were  aware  that  PIP  was  being  used,  but 
they  did  not  think  that  it  was  distracting,  or  that  it  interfered  with  seeing  the  instructor's  visuals. 

There  were  generally  small  differences  between  the  three  groups  on  most  of  the  items  shown 
it*  Figure  13.  However,  the  first  three  items  show  somewhat  higher  ratings  by  the  remote  students 
than  by  the  local  students.  Only  these  first  three  questions  produced  a  significant  group  effect  by  a 
one-way  analysis  of  variance  (ANOVA),  which  was  followed  by  Tukey  HSD  tests  to  test  for 
significant  differences  between  pairs  of  groups.  The  quality  of  the  presentation  was  judged 
significantly  higher  by  the  VTT  remote  students  seeing  the  alternating  PIP  than  the  VTT  local 
students,  F(2,65)  =  3.79,  p  <  .05.  For  the  other  two  questions,  the  VTT  regular  PIP  use  group  was 
significantly  higher  than  the  VTT  local  group  with  regard  to  seeing  the  instructor  and  visuals  at  the 
same  time,  F(2,65)  =  3.81,  p  <  .05,  and  with  regard  to  the  appropriateness  of  PIP,  F(2,65)  =  3.97, 
p  <  .05. 
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Question: 


Disagree  Agree 

1  2  3  4  5 


The  quality  of  the  presentation  was 
improved  by  using  PIP 


PIP  was  beneficial  in  allowing  both  the 
instructor  and  the  visuals  to  be  seen  at 
the  same  time 

PIP  was  used  appropriately 


The  PIP  image  was  clear 


There  were  usually  technical  problems 
in  using  PIP 

I  was  generally  unaware  that  PIP  was 
being  used 


Showing  the  instructor  in  the  PIP  was 
distracting 


The  PIP  interfered  with  seeing  the 
instructor’s  visuals 


m  VTT  Local  •  Alternating  PIP 
I  I  VTT  Remote  -  Alternating  PIP 
V/A  VTT  Remote  -  Regular  PIP  Use 


Figure  13.  Picture-in-picture  questionnaire  rating  responses. 


Figure  14  shows  student  perceptions  of  the  effectiveness  of  PIP  in  providing  a  rating  on  each 
of  two  questions:  ‘Training  was  effective  with  PIP”  and  “Training  was  effective  without  PIP.” 
Ratings  for  the  training  effectiveness  of  PIP  were  near  the  “[4]  Agree”  region  of  the  scale  and 
ratings  of  training  without  PIP  were  near  the  “[3]  neither  agree/disagree”  middle  region  of  the 
scale.  The  remote  students  show  a  greater  spread  between  ratings  on  the  two  questions,  with  the 
PIP  presentation  being  rated  slightly  higher  and  the  presentation  without  PIP  being  rated  slightly 
lower  than  the  ratings  given  by  local  students.  A  two-factor  ANOVA  was  performed  with  the  three 
treatment  groups  as  a  between-subjects  factor  and  the  two  rating  questions  as  a  within-subjects 
factor.  A  significant  main  effect  for  the  two  questions  indicated  that  ratings  were  higher  for  PIP 
than  without  PIP,  F(l,65)  =  38.01,  p<.01.  There  was  also  a  significant  interaction  between  the  two 
questions  and  three  treatment  groups,  F(2,65)  =  3.25,  p<.05,  but  the  main  effect  for  treatment 
groups  combining  the  two  questions  was  not  significant  as  a  consequence  of  the  interaction  pattern 
shown,  F(2,65)  =  1.07,  p>.05.  Relative  to  the  local  students,  the  remote  students  discriminated 
between  the  two  presentations  to  a  greater  extent.  Tukey  HSD  tests  indicated  that  the  mean 
difference  between  ratings  with  and  without  PIP  was  not  significant  for  the  VTT  local  group  (0.5 1), 
but  this  difference  was  significant  (p<.05)  for  both  the  alternating  PIP  (1.40)  and  regular  PIP  use 
(1.18)  VTT  remote  groups.  Thus,  the  difference  or  spread  between  ratings  of  effectiveness  with  and 
without  PIP  was  over  twice  as  large  for  VTT  remote  students  than  it  was  for  VTT  local  students. 
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Traininq  Was  Effective; 


Local  Remote  Remote 


- Alternating  PIP -  Regular  PIP  Use 


100 


Which  Presentation  Did  You  Prefer: 
■  With  PIP 

0  Without  PIP  92.9  93.3 


Local  Remote 

—  Alternating  PIP - 


Remote 

Regular  PIP  Use 


Figure  14.  Picture-in-picture  questionnaire  responses 
for  effectiveness  and  preference. 


Figure  14  shows  that  the  three  treatment  groups  differed  substantially  in  their  preferences  on 
the  two-choice  question  asking  “Which  presentation  did  you  prefer  the  most?”,  “When  PIP  was 
used  or  When  PIP  was  not  used.”  The  VTT  local  students  showed  only  a  slight  preference  for  a 
presentation  with  PIP  (54.3%)  compared  to  without  PIP  (45.7%).  By  contrast,  both  VTT  remote 
groups  had  a  strong  preference  for  the  PIP  presentation  (about  93%)  over  not  using  PIP  (about  7%). 
i^Chi  Square  test  showed  that  the  response  distributions  differed  significantly  (Yates  corrected 
X  (2)  =  9.06,  p  <  .05).  (One  or  two  students  per  group  did  not  answer  the  question,  and  if  they  were 
included  in  calculating  percent  preference  then  the  preference  for  PIP  would  be  reduced  to  51.4, 
86.6  and  82.3  percent  in  the  alternating  local  and  remote,  and  regular  PIP  remote  groups 
respectively). 

Although  the  VTT  remote  students  in  the  “Regular  PIP  Use”  group  did  not  see  alternating 
periods  with  and  without  PIP,  their  responses  were  generally  similar  to  the  alternating  PIP  remote 
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site  students.  Their  interpretation  of  the  question  on  when  PIP  was  not  used  may  have  been  judged 
relative  to  the  periods  when  just  the  instructor  was  shown  full  screen  in  comparison  to  the  use  of 
PIP  when  the  instructor  showed  his  visuals. 

Overall,  these  results  indicate  that  it  is  desirable  to  enable  PIP  on  the  outgoing  video  to  remote 
site  students  and  that  it  is  probably  not  necessary  for  local  site  students.  The  magnitude  of  the 
preference  for  PIP  shown  here  may  reflect  the  long  periods  during  which  visuals  are  shown  on  the 
outgoing  video  in  this  course  and  might  not  be  expected  to  be  as  large  in  other  courses  using  visuals 
less  extensively.  Records  kept  during  the  first  two  days  of  the  alternating  PIP  portion  of  this  study 
indicate  a  substantial  amount  of  time  during  which  visuals  are  shown  and  the  instructor  could  be 
off  screen.  Celestial  navigation  visuals  and  strip  forms  were  found  to  be  shown  an  average  of  about 
10  minutes  at  a  time  and  to  comprise  65%  of  the  total  instruction  time  (excluding  breaks, 
homework  laboratories,  and  the  film). 

Electronic  Presentation  Program  Evaluation 

A  five  item  questionnaire  was  administered  to  students  that  asked  them  to  compare  a 
computerized  slide  presentation  lesson  using  a  strip  form  with  the  conventional  hand-written 
method  used  in  other  lessons  (see  Appendix  C).  Four  items  were  rated  on  the  “[1]  strongly 
disagree”  to  “[5]  strongly  agree”  five  point  scale,  and  one  item  offered  a  two-altemative  choice 
between  the  two  methods. 

The  questionnaire  was  completed  by  23  students  (15  local  and  8  remote)  during  one  class 
convening.  Data  for  local  and  remote  students  were  combined  because  they  responded  virtually  the 
same,  differing  by  less  than  two  tenths  of  one  rating  unit.  The  four  rating  items  were  all  rated  in  the 
“[4]  agree”  region  of  the  scale. 

Two  questionnaire  items  asked  for  a  rating  on  whether  “Training  was  effective  with  the  method 
that  used,”  in  one  item,  “the  TV  copy  stand  with  paper  and  hand-writing  using  a  marker  pen,”  and 
in  the  other  item,  “the  computerized  electronic  presentation  program.”  A  rating  of  4.17  was  given 
to  the  hand  written  method  and  a  rating  of  3.96  was  given  to  the  computerized  electronic 
presentation  program.  This  difference  was  only  0.21  of  a  rating  unit  in  favor  of  the  conventional 
method. 

Another  pair  of  questions  asked  for  separate  ratings  of  the  readability  of  text  with  the  two 
presentation  methods.  An  average  rating  of  4.32  was  given  to  the  item  asking  if  “Text  was  readable 
with  hand-written  method.”  An  average  rating  of  4.26  was  given  to  the  item  asking  if  ‘Text  was 
readable  with  computerized  presentation.”  This  difference  was  only  0.06  of  a  rating  unit  in  favor 
of  the  conventional  method. 

Student  preference  for  the  two  presentation  methods  was  asked  with  the  question  “Which 
presentation  method  do  you  prefer  the  most?”  There  were  two  response  alternatives:  (a)  “TV  copy 
stand  with  paper  and  hand-writing  with  a  marker,”  or  (b)  “Electronic  presentation  program.” 
Omitting  the  3  students  that  did  not  respond,  the  hand-written  method  on  the  TV  copy  stand  was 
chosen  by  55%  of  the  students  and  the  electronic  presentation  program  was  chosen  by  the  other 
45%.  This  slight  difference  was  not  significant  by  a  Chi  Square  test. 

The  results  indicate  that  there  was  essentially  no  difference  in  effectiveness  or  readability  for 
either  method  and  that  students  had  little  preference  for  using  one  method  or  the  other.  The 
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questionnaire  was  not  administered  to  additional  classes  because  the  obtained  differences  were  so 
slight  with  this  initial  sample  of  students. 


Discussion 

The  adaptation  of  the  Celestial  Navigation  refresher  course  for  delivery  by  video-teletraining 
to  remote  students  has  been  successful.  The  course  has  been  regularly  offered  to  remote  VTT  sites 
from  San  Diego  without  any  significant  problems.  TTiis  method  of  delivery  is  accepted  by  students 
and  instructors,  and  the  overall  direct  experience  of  the  instructors  has  been  positive.  The  initial 
concerns  with  the  feasibility  of  delivering  the  course  were  overcome  as  a  result  of  several  course 
conversion  techniques  and  the  efforts  of  the  instructors  to  monitor  remote  site  students.  The  initial 
concerns  with  the  feasibility  of  teaching  the  course  were  with  the  variety  of  small  print  visuals 
u^d,  physical  space  for  plotting,  and  whether  the  computational  content  would  be  a  problem  in 
distant  student  laboratories.  There  were  only  slight  trends  in  the  evaluation  results  in  the  direction 
of  these  concerns.  Interaction  was  at  sufficient  levels  and  the  VTT  sites  were  able  to  communicate 
on  performing  the  computations  and  on  clarifying  problems.  The  principal  findings  from  the 
evaluation  are  discussed  below  in  terms  of  student  performance,  student  evaluations,  observed 
interaction,  and  course  conversion  experiences. 

Student  Performance 

Student  performance  in  all  groups  was  generally  at  a  high  level  over  the  three  performance 
measures.  Officer  students  scored  at  a  higher  level  than  did  enlisted  students  and  so  the  groups  were 
examined  separately  because  the  enrollment  pattern  in  remote  classrooms  resulted  in  many  more 
enlisted  students  (75%)  than  in  local  and  traditional  classrooms  (44-38%).  Remote  students  were 
also  slightly  more  junior  than  those  at  local  sites. 

There  was  no  significant  difference  between  the  treatment  groups  in  terms  of  the  percentage  of 
student  passing  the  course  (which  required  75%  on  both  the  homework  and  the  final  examination). 
The  average  of  the  13  homework  assignments  during  the  week  was  very  similar  among  the  three 
treatment  groups  and  did  not  differ  significantly.  There  was  a  small  but  significant  difference 
between  the  groups  on  the  final  examination  where  scores  were  lower  at  remote  sites  than  at  the 
local  site  by  4.4%  for  officer  students  and  5.1%  for  enlisted  students.  Statistically  controlling  for 
the  inequities  in  student  characteristics  resulting  from  the  enrollment  pattern  at  remote  sites 
indicated  a  smaller  adjusted  difference  between  local  and  remote  sites  of  4.0%  when  officer- 
enlisted  and  military  seniority  were  controlled.  Previous  studies  have  found  the  difference  between 
remote  and  local  sites  to  range  from  no  difference  to  five  percent  (Simpson,  1990,  1991a,  1991b, 
1992,  1993,  1995;  and  studies  reviewed  in  Wetzel,  et  al.,  1994,  pp  19-21).  Although  the  rate  of 
passing  the  course  was  not  affected  because  performance  was  still  above  criterion,  the  small 
difference  on  the  final  examination  stresses  the  importance  of  continued  monitoring  of  remote 
student  comprehension  during  the  course.  The  possibility  that  some  of  the  lower  performance  is 
due  to  the  plotting  portion  of  the  examination  could  be  examined  in  future  classes. 

Student  Attitudes  Toward  Tkaining 

Student  responses  on  the  rating  items  were  consistently  higher  for  VTT  local  and  traditional 
students  than  for  VTT  remote  students.  However,  the  magnitude  of  the  differences  between  the 
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treatment  groups  was  generally  not  large  and  ratings  for  all  groups  were  generally  high  and  in  a 
positive  direction.  The  pattern  and  magnitude  of  differences  observed  in  this  study  are  typical  of 
that  shown  in  previous  VTT  research  (Simpson,  et  al.,  1990,  1991a,  1991b,  1992,  1993,  1995; 
Wetzel,  et  al.,  1993, 1994, 1995).  Group  differences  were  negligible  for  topics  concerned  with  class 
participation,  audio-visual  factors  related  to  hearing  and  seeing  the  instruction,  and  instructor 
presentations  of  material  (a  third  or  less  of  a  rating  unit  group  difference  on  the  five  point  scale). 
These  results  indicated  that  the  various  small  print  publications  and  strip  forms  were  successfully 
adapted  for  delivery  by  VTT. 

Remote  students  gave  ratings  only  about  a  half  a  unit  lower  than  local  students  for  most  items 
related  to  getting  help,  assistance,  feedback  or  attention.  However,  there  was  no  significant 
difference  between  groups  in  rating  either  the  difficulty  or  pace  of  the  course.  Remote  students 
were  more  likely  than  local  students  to  indicate  that  they  had  less  access  to  the  instructor  by  a  full 
rating  unit  and  were  about  30%  more  likely  to  say  they  had  fewer  opportunities  to  interact  with  the 
instructor.  Remote  site  students  were  also  a  little  more  likely  to  cite  other  students  as  a  source  of 
assistance  outside  of  class.  However,  remote  students  were  willing  to  take  another  VTT  course  and 
their  preferences  for  a  method  of  instruction  for  this  course  were  not  less  favorable  than  local 
students.  Direct  experience  with  the  impact  of  VTT  appeared  to  increase  the  extent  of  student 
acceptance  of  VTT-remote  students  were  more  accepting  of  VTT  than  traditional  students  without 
the  direct  VTT  experience. 

Facilities  related  items  reflected  conditions  during  the  study  period,  such  as  access  to  the  room 
and  confined  table  space  for  materials  at  some  facilities,  but  the  use  of  charts  was  not  particularly 
a  problem.  Providing  larger  rooms  and  tables  to  better  accommodate  this  and  other  courses  with 
hands-on  laboratories  should  be  included  in  future  VTT  site  planning. 

Interaction  and  Participation 

The  tally  of  interactions  in  the  course  was  recorded  to  assess  the  relative  level  of  instructor- 
student  interaction  among  local  and  remote  sites  and  to  assess  whether  the  level  of  participation  by 
remote  student  was  adequate.  The  results  with  this  objective  measure  indicated  that  there  were 
adequate  levels  of  remote  site  student  participation  and  instructor  interaction  with  remote  students. 

Most  of  the  interactions  during  class  sessions  were  initiated  by  instructors  in  questioning  that 
focused  on  answers  to  steps  in  the  computations  and  assessing  other  knowledge.  Instructors  used 
a  mix  of  questions  that  were  left  open  to  any  site,  directed  at  a  site,  or  directed  at  individual 
students.  The  mixture  of  instructor  questions  would  be  judged  beneficial  because  the  instructor  can 
assess  knowledge  levels  in  both  those  students  who  are  likely  to  respond  when  they  know  an 
answer  and  those  individuals  who  are  less  likely  to  participate. 

Despite  their  fewer  numbers,  remote  site  students  were  not  disadvantaged  in  terms  of  the  level 
of  interaction  observed.  There  tended  to  be  fewer  interactions  per  hour  associated  with  the  remote 
sites  because  there  were  fewer  remote  than  local  site  students.  When  expressed  in  terms  of 
interactions  per  student  per  hour,  remote  site  students  received  a  significantly  higher  rate  of 
instructor  questions  associated  with  their  site  than  did  local  students. 

There  was  a  tendency  for  both  student  and  instructor  interaction  to  increase  over  days.  In 
particular,  student  initiated  questions  increased  after  the  first  day  and  unanswered  instructor 
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questions  declined  after  the  first  day.  Local  site  students  overcame  their  tendency  to  speak  directly 
to  the  instructor  as  the  week  progressed  because  there  were  fewer  instructor  reminders  to  use  their 
microphones  so  remote  sites  could  hear  them. 

Technology  Evaluations 

Picture-in-Picture  (PIP)  technology  was  found  beneficial  for  showing  both  the  instructor  and 
visuals  such  as  strip  forms  during  periods  when  only  the  visuals  and  not  the  instructor  would  have 
normally  been  shown.  The  use  of  PIP  was  generally  favored  by  remote  site  students.  VTT  remote 
students  gave  higher  ratings  with  regard  to  the  quality  of  the  presentation  with  PIP  and  the 
difference  between  ratings  of  effectiveness  with  and  without  PIP  was  also  larger.  When  asked  for 
their  preference,  remote  students  overwhelmingly  chose  the  presentation  with  PIP  over  the 
presentation  without  PIP,  while  local  students  were  relatively  neutral  in  their  preference.  This 
technique  could  be  used  for  the  video  sent  out  to  remote  students  in  other  courses  where  an 
instructor  might  be  off-screen  for  lengthy  periods. 

A  brief  questionnaire  also  asked  students  to  compare  an  electronic  presentation  program  lesson 
with  the  conventional  hand-written  method  used  in  other  lessons.  Students  had  no  preference  for 
using  one  method  or  the  other  and  gave  the  methods  similar  effectiveness  and  readability  ratings. 
Thus,  the  choice  of  these  technologies  can  be  based  on  practical  and  efficiency  factors. 

VTT  Course  Conversion 

Several  aspects  of  the  experience  with  the  Celestial  Navigation  course  are  relevant  to  adapting 
other  courses  to  VTT.  The  adaptation  of  the  course  required  a  concerted  planning  phase  to  address 
potential  difficulties,  revising  visual  materials,  procuring  remote  site  materials,  developing  new 
instructor  behaviors  appropriate  to  VTT,  and  developing  support  mechanisms  at  remote  sites  (these 
topics  are  discussed  in  a  useful  course  conversion  guide  developed  for  the  Navy  by  Simpson 
1993). 

Instructor  transparencies  must  typically  be  revised  to  contain  fewer  lines  and  words,  put  in  a 
landscape  format  with  a  consistent  size,  and  reproduced  on  paper  to  avoid  reflections.  The 
approach  to  the  visuals  in  this  course  addressed  a  problem  with  video  displays  of  the  many  small 
print  tables  in  nautical  publications.  Showing  an  entire  table  rendered  the  small  print  unreadable 
and  zooming  in  on  relevant  text  lost  the  overall  context  of  relevant  column  and  row  heading.  The 
critical  portions  of  text  were  enlarged  and  arrows  indicated  where  these  entries  were  located  on  the 
page  so  that  students  could  locate  entries  in  their  own  pubhcations.  Lengthy  strip  forms  were  also 
enlarged  and  segmented  into  several  pages. 

Remote  site  facilitators  play  an  important  role  as  an  extension  of  the  instructor  for  students  at 
a  distance.  An  important  principle  illustrated  in  this  course  was  the  development  of  scoring 
methods  that  captured  the  expertise  of  the  instructors  and  allowed  facilitators  to  act  as  an  extension 
of  the  instructor.  Facilitators  were  able  to  score  student  plotting  charts  by  using  the  acetate  overlay 
marked  with  the  correct  course,  fixes  and  labeling  required  of  students.  A  daily  schedule  of  events 
is  another  particularly  beneficial  mechanism  that  allows  facilitators  to  assist  the  instructor  when 
there  are  many  materials  to  be  distributed  at  appropriate  times  or  when  laboratory  equipment  is  to 
be  setup. 
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Courses  that  depart  from  the  lecture-based  courses  typically  taught  by  VTT  require  additional 
attention.  Courses  with  special  requirements,  such  as  laboratories  where  students  perform  their 
own  work,  should  be  monitored  in  order  to  ensure  that  a  high  quality  VTT  version  of  the  course 
continues  to  be  maintained.  As  remote  site  facilitators  and  instructors  rotate  in  their  assignments, 
the  training  and  practical  experience  bearing  on  the  special  concerns  of  the  cotirse  need  to  be 
conveyed  to  their  replacements. 

Classroom  facilities  also  require  attention  in  delivering  a  wider  range  of  VTT  courses  that 
include  laboratories.  VTT  classrooms  are  used  by  a  variety  of  courses  and  are  not  dedicated  to  the 
materials  and  equipment  for  just  one  course.  The  key  to  reusing  the  same  room  for  courses  with 
special  requirements  is  to  develop  portable  methods  to  allow  materials  and  equipment  to  be  moved 
in  and  out  of  the  room.  Laboratory  courses  will  also  generally  require  somewhat  larger  rooms  and 
table  space  adequate  to  the  student  materials.  As  noted  in  this  study,  some  sites  had  small  tables 
that  made  it  more  difficult  to  use  the  four  publications  and  various  small  items  used  in  plotting. 
Other  courses  with  more  substantial  equipment  may  require  more  classroom  space,  extra  storage 
space  between  class  convenings,  and  require  enhanced  electrical  power.  Although  the  Celestial 
Navigation  course  was  a  relatively  mild  form  of  a  laboratory  course,  the  experience  with  this  and 
other  subsequent  laboratory  courses  has  suggested  the  importance  of  facilities  and  portability 
considerations  for  other  future  laboratory  VTT  courses. 

Finally,  concern  for  remote  site  student  comprehension  and  participation  is  an  issue  common 
to  a  variety  of  VTT  courses.  As  mentioned  previously,  instructors  in  this  course  consistently  used 
a  variety  of  questioning  techniques  to  make  periodic  verbal  checks  on  student  comprehension.  A 
student  roster  forwarded  from  the  remote  site  is  beneficial  in  allowing  the  instructor  to  direct 
questions  to  individual  students,  particularly  those  who  are  less  hkely  to  participate.  An  adequate 
level  of  remote  site  student  initiated  participation  was  also  observed  in  this  course.  The  success  of 
the  Celestial  Navigation  course  was  related  to  the  development  of  effective  instructor  behaviors  in 
attending  to  remote  site  students  and  having  learned  to  use  a  variety  of  technologies  to  delivery 
effective  presentations.  The  experience  with  this  course  also  highlights  the  importance  of  tight 
coordination  and  communication  between  instructors  and  remote  site  facilitators. 


Recommendations 

The  following  recommendations  are  for  the  Chief  of  Naval  Education  and  Training  and  the 
CNET  Electronic  Schoolhouse  Network. 

1.  Delivery  of  the  Celestial  Navigation  refresher  course  by  videoteletraining  has  been  suc¬ 
cessful  and  the  use  of  this  method  should  be  expanded  to  other  sites  as  appropriate  to  satisfy 
demand  for  the  course. 

2.  VTT  courses  with  atypical  requirements  such  as  student  laboratories  should  be  given  spe¬ 
cial  attention  to  maintain  a  high  quality  VTT  version  of  the  course.  Such  attention  includes  moni¬ 
toring  remote  site  student  comprehension,  conveying  VTT  lessons  learned  as  remote  site 
facilitators  and  instructors  rotate  in  their  assignments,  and  providing  sufficient  space  for  the  addi¬ 
tional  instructional  materials  found  in  this  and  other  future  laboratory  courses. 
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Appendix  A 

Celestial  Navigation  Student  Questionnaire 


A-O 


CELESTIAL  NAVIGATION  STUDENT  QUESTIONNAIRE 


a .  Name _  _  _  _ 

Last  First  MI  Rank 

b.  Today's  date _  c.  Location  [  ]  San  Diego 

[  ]  Treasure  Island 
[  ]  Bangor 


For  each  of  the  statements  followed  by  [1]  [2]  [3]  [4]  [5]  ,  please  check 

the  box  indicating  how  well  you  agree  with  the  statement  using  this  scale: 

[1]  [2]  [3]  [4]  [5] 

Strongly  Disagree  Neither  Agree  Strongly 

Disagree  Agree/  Agree 

Disagree 


INSTRUCTORS 

1.  Instructors  presented  lessons  clearly 

2.  Instructors  encouraged  class  participation 

3 .  Instructors  answered  student  questions  adequately 

4.  Instructors  could  be  seen  clearly 

5 .  Instructors  could  be  heard  adequately 

6.  Instructors  helped  students  who  needed  assistance 


LEARNING  &  CLASSROOM  ACTIVITIES 

7 .  Concepts  or  theory  underlying  the  computations 
were  adequately  presented 

8 .  Computational  procedures  were  adequately  presented 

9 .  I  hesitated  to  ask  questions  in  order  to  clarify 
concepts  or  steps  in  the  computations 

10.  I  received  help  on  computational  problems  when 
I  needed  it 

11.  I  had  access  to  the  classroom  when  class 
was  not  in  session 

12 .  I  had  access  to  instructors  outside  of  class 

13 .  Assigned  problems  were  graded  to  provide  feedback 


Disagree  Agree 


SD 

D 

N 

A 

SA 

[1] 

[2] 

[3] 

[4] 

[5] 

[1] 

[2] 

[3] 

[4] 

[5] 
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[4] 

[5] 

[1] 
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[3] 

[4] 

[5] 

[1] 

[2] 

[3] 

[4] 

[5] 
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[2] 

[3] 

[4] 
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Disagree 

Agree 
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SA 
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[1] 

[2] 

[3] 

[4] 

[5] 

[1] 

[2] 

[3] 

[4] 

[5] 

A-l 


TRAINING  AIDS 


14.  Training  aids  were  used  effectively 

15.  Details  of  training  aids  could  be  clearly  seen' 

16.  Instructor's  overheads /slides  helped  me  locate 
relevant  entries  in  the  publications 

17.  Instructor's  visual  aids  for  strip  forms 
worked  effectively 

18.  Instructor's  drawings  and/or  writing  during 
class  were  clear  and  understandable 

19.  The  film/videotape  was  informative 

20.  The  film/videotape  was  clearly  seen  and  heard 

21.  Table  space  was  adequate  to  accommodate 
publications,  charts,  and  to  perform  plotting 

22.  Charts  and  publications  were  used  without  problems 


INTERACTION/ PARTICIPATION 

23.  Interaction  between  instructors  and  students 
was  sufficient  to  support  learning  objectives 

24.  Class  participation  was  sufficient  to  support 
learning  objectives 

25.  Students  aided  one  another  in  performing 
the  computations 

26.  Instructor  gave  equal  attention  to  all  students 


OVERALL 

27.  Instructor (s)  compare  favorably  with  the  best 
Navy  instructors  I  have  had  in  the  past 

28.  This  course  compares  favorably  with  the  best 
Navy  courses  I  have  taken  in  the  past 

29.  I  found  the  course  material  to  be  difficult 


Disagree 
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[1]  [2]  [3] 

Agree 

A  SA 

[4]  [5] 

[1] 

[2] 

[3] 

[4j 

'  [5] 

[1] 

[2] 

[3] 

[4] 

[5] 

[1] 

[2] 

[3] 

[4] 

[5] 

[1] 

[2] 

[3] 

[4] 

[5] 

[1] 

[2] 

[3] 

[4] 

[5] 

[1] 

[2] 

[3] 

[4] 

[5] 

[1] 

[2] 

[3] 

[4] 

[5] 

[1] 

[2] 

[3] 

[4] 

[5] 

Disagree 

SD  D  N 

Agree 

A  SA 

[1] 

[2] 

[3] 

[4] 

[5] 

[1] 

[2] 

[3] 

[4] 

[5] 

[1] 

[2] 

[3] 

[4] 

[5] 

[1] 

[2] 

[3] 

[4] 

[5] 

Disagree 

SD  D  N 

Agree 

A  SA 

[1] 

[2] 

[3] 

[4] 

[5] 

[1] 

[2] 

[3] 

[4] 

[5] 

[1] 

[2] 

[3] 

[4] 

[5] 
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30-  The  pace  of  the  course  was: 

[  ]  [  ]  [  ]  [  ]  [  ] 

Too-Slow  Somewhat-Slow  About-right  Somewhat-Fast  Too-Fast 


BACKGROUND  Disagree  Agree 

SD  D  N  A  SA 

31.  Before  this  course,  I  was  proficient  at  manually 

performing  the  celestial  navigation  computations  [1]  [2]  [3]  [4]  [5] 

32.  I  previously  had  experience  in  manually  performing 
the  celestial  navigation  computations  in  this  course; 

[  ]  No 

[  ]  Yes,  on-the-job  experience 
[  ]  Yes,  course  work 

[  ]  Yes,  both  on-the-job  experience  and  course  work 
[  ]  Other  (please  explain) ;  _ 

33.  Will  your  duties  or  next  assignment  directly  involve  application 
of  skills  from  this  course; 

[  ]  No 

[  ]  Yes,  infrequently  for  shipboard  duties 
[  ]  Yes,  routinely  for  shipboard  duties 
[  ]  The  course  was  primarily  for  advancement  purposes 
[  ]  Other  (please  explain) :  _ 


VIDEO-TELETRAINING  Disagree  Agree 

SD  D  N  A  SA 

34.  Instructors  coordinated  activities  among  classes 

effectively  [1]  [2]  [3]  [4]  [5] 

35.  Local  and  remote  sites  participated  equally  in 

the  class  [1]  [2]  [3]  [4]  [5] 

36.  The  instructor  called  on  students  at  all 

of  the  sites  about  equally  [1]  [2]  [3]  [4]  [5] 

37.  The  VTT  facilitators  assisting  the  instructor 

were  helpful  to  students  in  the  class  [1]  [2]  [3]  [4]  [5] 

38.  Who  most  frequently  provided  assistance  to  you 
outside  of  regular  class  hours: 

[  ]  Instructor 
[  ]  Facilitator 
[  ]  Other  students 

[  ]  Other  (please  explain) :  _ 

39.  Which  method  of  instruction  would  you  prefer  for  this  course: 

[  ]  Video  Tele-Training  (VTT) 

[  ]  Traditional  method  (non-VTT) 

[  ]  Either  method 
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40,  How  did  the  VTT  method  of  instruction  affect  your  opportunities  to 
interact  with  the  instructor? 

[  ]  more  opportunities 
[  ]  no  effect  on  opportunities 
[  ]  fewer  opportunities 

41.  If  you  had  a  choice,  would  you  take  another  VTT  course? 

[  ]  Yes 
[  ]  No 

Please  explain: _ 


COMMENTS:  Please  answer  the  following  questions  by  writing  in  your 

comments  on  the  blank  lines. 


42.  What  did  you  like  the  most  about  this  course?, 


43,  What  did  you  like  the  least  about  this  course?, 


44.  Discuss  any  suggestions  you  have  for  improving  the 


course 


Appendix  B 

Picture-in-Picture  Questionnaire 


B-0 


EVALUATION  OF  PIP 


Picture-in-Picture  (PIP)  was  used  during  some  of  your  lectures  to  show  a 
second  image  inserted  within  a  window  in  the  larger  image  on  the  screen. 
For  example,  one  image  showed  the  instructor  and  the  other  showed  training 
aids  such  as  graphics,  slides  or  writings  made  by  the  instructor.  Please 
evaluate  this  technique  in  terms  of  its  effectiveness  and  your  preference. 


For  each  of  the  statements  followed  by  [1]  [2]  [3]  [4]  [5]  ,  please  check 

the  box  indicating  how  well  you  agree  with  the  statement  using  this  scale: 

[1]  C2]  [3]  [4]  [5] 

Strongly  Disagree  Neither  Agree  Strongly 

Disagree  Agree/  Agree 

Disagree 


Check  Site:  [  ]  San  Diego  [  ]  Treasure  Island  [  ]  Bangor 

Date :  _ _ _ 

Name  of  Course: 

Disagree  Agree 


1. 

The  quality  of  the  presentation  was  improved  by  using  PIP 

SD 

[1] 

D 

[2] 

N 

[3] 

A 

[4] 

SA 

[5] 

2. 

The  PIP  interfered  with  seeing  the  instructor's  visuals 

[1] 

[2] 

[3] 

[4] 

[5] 

3  . 

I  was  generally  unaware  that  PIP  was  being  used 

[1] 

[2] 

[3] 

[4] 

[5] 

4. 

Showing  the  instructor  in  the  PIP  was  distracting 

[IJ 

[2] 

[3] 

[4] 

[5] 

5. 

PIP  was  beneficial  in  allowing  both  the  instructor 
and  the  visuals  to  be  seen  at  the  same  time 

[1] 

[2] 

[3] 

[4] 

[5] 

6. 

There  were  usually  technical  problems  in  using  PIP 

[1] 

[2] 

[3] 

[4] 

[5] 

7. 

PIP  was  used  appropriately 

[1] 

[2] 

[3] 

[4] 

[5] 

8. 

The  PIP  image  was  clear 

[1] 

[2] 

[3] 

[4] 

[5] 

9. 

Training  was  effective  WITH  PIP 

[1] 

[2] 

[3] 

[4] 

[5] 

10 

.  Training  was  effective  WITHOUT  PIP 

[1] 

[2] 

[3] 

[4] 

[5] 

11.  Which  presentation  did  you  prefer  the  most  ? 
[  ]  When  PIP  WAS  used 
[  ]  When  PIP  WAS  NOT  used 

Please  comment  on  the  use  of  PIP  in  the  course: 


B-1 


Appendix  C 

Electronic  Presentation  Program  Questionnaire 


C-0 


Electronic  Presentation  Program  Questionnaire 
(Addendum  to  Celestial  Navigation  Questionnaire) 


Your  instructor  used  two  different  presentation  technologies  to  show 
his  graphics  and  strip  forms: 

(1)  A  conventional  hand-written  method  using  paper  on  the  TV  copy  stand, 
e.g.,  he  wrote  on  a  paper  strip  form  with  a  marker. 

(2)  A  computerized  electronic  presentation  program  (VideoShow) , 

e.g.,  entries  in  the  electronic  stripform  were  revealed  one  at  a  time. 


Disagree  Agree 
SD  D  N  A  SA 

A.  Training  was  effective  with  the  method  that  used: 
the  TV  copy  stand  with  paper  and  hand-writing 

using  a  marker  pen.  [1]  [2]  [3]  [4]  [5] 


B.  Training  was  effective  with  the  method  that  used: 

the  computerized  electronic  presentation  program.  [1]  [2]  [3]  [4]  [5] 


C.  Which  presentation  method  do  you  prefer  the  most: 

(please  pick  only  one) 

[  ]  TV  copy  stand  with  paper  and  hand-writing  with  a  marker 
[  ]  Electronic  presentation  program  (VideoShow) 


Text  was  readable 

with 

hand-written 

method 

[1] 

[2] 

[3] 

[4] 

[5] 

Text  was  readable 

with 

computerized 

presentation 

[1] 

[2] 

[3] 

[4] 

[5] 

Please  provide  any  comments  on  these  two  methods: 
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Appendix  D 

Student  Groups  and  Performance  Summary  Tables 


D-O 


Table  D-1 

Number  of  Students  Per  Measure  and  Average  Military  Rank 


Treatment  Group 


Group  and  Measure  VTT  Local  VTT  Remote*  Traditional  Prior  Traditional 


Student  Grades: 


Number  of  Officer  Students 

76 

16 

48 

48 

Mean  Military  Rank** 

2.7 

2.4 

2.7 

2.5 

Median  Military  Rank 

3.0 

3.0 

3.0 

3.0 

Number  of  Enlisted  Students 

59 

50 

30 

43 

Mean  Military  Rank** 

5.5 

4.8 

4.1 

5.6 

Median  Military  Rank 

6.0 

5.0 

4.0 

6.0 

Total  Number  of  Students 

135 

66 

78 

91 

Number  of  Classes 

9 

9 

5 

5 

Students  Completing  Questionnaire: 

Number  of  Students 

128 

61 

52 

- 

Number  of  Classes 

9 

9 

3 

- 

Students  in  Interaction  Tally: 

Number  of  Students 

101 

46 

- 

- 

Number  of  Classes 

6 

6 

- 

_ 

*  VTT  remote  students  were  primarily  at  Treasure  Island.  Of  the  total  shown,  the  Bangor 
remote  site  contributed  12  students  in  5  classes  for  the  grades,  9  students  in  3  classes 
for  questionnaires,  and  10  students  in  3  classes  for  the  interaction  tally. 

**Military  rank  for  Officers  ranged  from  0-1  to  0-5;  Enlisted  ranged  from  E-2  to  E-8. 

Note:  Treatment  group  data  were  collected  during  the  following  time  periods: 

Nine  VTT  classes  between  June  1993  and  August  1994. 

Five  traditional  classes  were  May,  June,  and  July  1993,  and  March  and  May  1994 
(questionnaires  were  administered  in  the  last  three  traditional  classes). 

Five  prior  traditional  were  April,  March,  February,  and  Jan  1993,  and  Dec  1992. 
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Table  D-2 


Homework  Average  Percent  Correct, 
Standard  Deviations,  and  Number  of  Students 


Treatment  Group 

Group  and  Measure 

VTT  Local 

VTT  Remote 

Traditional 

Prior  Traditional 

Officer: 

Mean 

95.86 

95.55 

96.50 

95.54 

Standard  Deviation 

3.74 

3.47 

3.21 

4.22 

Number  of  Students 

76 

16 

48 

48 

Enlisted: 

Mean 

91.09 

92.38 

92.44 

92.98 

Standard  Deviation 

9.36 

5.77 

4.93 

8.61 

Number  of  Students 

59 

50 

30 

43 

Table  D-3 

Final  Examination  Average  Percent  Correct, 
Standard  Deviations,  and  Number  of  Students 

Treatment  Group 

Group  and  Measure 

VTT  Local 

VTT  Remote 

Traditional 

Prior  Traditional 

Officer: 

Mean 

96.69 

92.28 

95.52 

88.82 

Standard  Deviation 

3.19 

5.49 

5.18 

6.58 

Number  of  Students 

76 

16 

48 

48 

Enlisted: 

Mean 

92.47 

87.32 

90.86 

86.17 

Standard  Deviation 

9.17 

10.26 

11.25 

11.02 

Number  of  Students 

59 

49 

30 

43 

Combined  Officer  +  Enlisted: 

Actual  Mean 

94.85 

88.55 

93.73 

87.57 

Adjusted  Mean  (OE)* 

94.58 

89.70 

93.23 

_ 

Adjusted  Mean  (OE  &  Rank) 

94.34 

90.31 

93.14 

- 

*Note:  Adjusted  means  are  based  on  analysis  of  covariance  predictions  using 
Officer-Enlisted  status  (OE)  or  OE  status  and  Z-score  military  rank  as  covariates. 
Note:  Enlisted  VTT  remote  based  on  49  students  because  numeric  value  of 
one  failing  score  not  recorded  for  50  students. 
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Table  D-4 


Percentage  and  Number  of  Student  Passing  or  Failing 
the  Course,  Homework,  and  Final  Examination. 


Group  and  Measure 

Treatment  Group 

VTT  Local 

VTT  Remote 

Traditional 

Prior  Traditional 

Officer: 

Percent  Passing 

100.(X) 

100.00 

100.00 

100.00 

Percent  Failing 

0.00 

0.00 

0.00 

0.00 

Number  Passing 

76 

16 

48 

48 

Number  Failing 

0 

0 

0 

0 

Total  (Passing  +  Failing) 

76 

16 

48 

48 

Number  Failing  Homework 

0 

0 

0 

0 

Number  Failing  Final  Exam 

0 

0 

0 

0 

Number  Incomplete  * 

0 

0 

0 

0 

Enlisted: 

Percent  Passing 

93.22 

92.00 

96.67 

81.40 

Percent  Failing 

6.78 

8.00 

3.33 

18.60 

Number  Passing 

55 

46 

29 

35 

Number  Failing 

4 

4 

1 

8 

Total  (Passing  +  Failing) 

59 

50 

30 

43 

Number  Failing  Homework 

4 

0 

0 

1 

Number  Failing  Final  Exam 

3 

4 

1 

7 

Number  Incomplete  * 

4 

7 

2 

3 

*Note:  Incompletes  are  students  recalled  to  home  command  or  unauthorized 
absences  before  completion  of  course  and  are  not  counted  in  other  rows  above. 
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Appendix  £ 

Summary  Tables  for  Open-Ended  Questionnaire  Items 


E-O 


Table  E-1 


Percentage  of  Treatment  Group  Responses  to  Open-Ended  Question  Number  41 
"If  you  had  a  choice,  would  you  take  another  VTT  course  ?" 


Group  and  Comment  Category 

Treatment  Group 

VTT  Local 

VTT  Remote 

Combined  Groups 

Positive  Comment 

33.6 

36.1 

34.4 

Negative  Comment 

11.7 

9.8 

11.1 

Positive  and  Negative 

4.7 

11.5 

6.9 

No  Comment 

50.0 

42.6 

47.6 

Total  Percent 

100.0 

100.0 

100.0 

Total  Students 

128 

61 

189 

Table  E-2 

Percentage  of  Students  in  Each  Treatment  Group  That 

Offered  Comments  to  Open-Ended  Questions  (Items  42, 43,  and  44) 

Group  and  Question 

Treatment  Group 

Traditional 

Local 

Remote 

All  Groups 

42.  Liked  most  about  course 

86.5 

82.8 

77.0 

82.1 

43.  Liked  least  about  course 

53.8 

67.1 

67.2 

64.3 

44.  Suggestions  for  improvement 

44.2 

50.0 

60.6 

51.4 

Total  Number  of  Students 

52 

128 

61 

241 

Note:  percentages  of  total  students  that  gave  one  or  more  comments  to  a  question 
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Table  E-3 


Percentage  of  Treatment  Group  Responses  to  Open-Ended  Question 
"What  did  you  like  the  most  about  this  course?" 


Group  and  Comment  Category 

Treatment  Group 

Traditional 

Local 

Remote 

All  Groups 

a.  Subject  Matter 

21.7 

31.7 

26.1 

28.0 

b.  Pace  of  Course  Appropriate 

6.7 

7.8 

7.3 

7.4 

c.  VTT 

0.0 

3.5 

16.0 

5.9 

d.  Instructors 

21.6 

12.0 

7.2 

13.0 

e.  Exercises  and  Practice 

8.3 

4.9 

2.9 

5.2 

f.  Homework  and  Calculations 

5.0 

8.5 

4.3 

6.4 

g.  Instructional  Strategies 

18.3 

6.3 

4.3 

8.5 

h.  Instructional  Materials 

5.0 

4.2 

2.9 

4.1 

i.  Interaction 

0.0 

2.8 

1.5 

1.9 

j.  Miscellaneous 

1.7 

2.8 

7.2 

3.7 

No  Comment 

11.7 

15.5 

20.3 

15.9 

Total  Percent 

100.0 

100.0 

100.0 

100.0 

Number  of  1st  Comments 

45 

106 

47 

198 

Number  of  2nd  Comments 

8 

14 

8 

30 

Total  Comments 

53 

120 

55 

228 

Number  No  Comment 

7 

22 

14 

43 

Total  Comments  and  No  Comment 

60 

142 

69 

271 

Number  of  Students 

52 

128 

61 

241 

Note;  Divisor  for  percents  was  total  of  comments  and  no  comment 
a.  Subject  Matter  (help  with  job,  need  to  know,  valuable  skill,  theory,  review. 


interesting,  challenging). 

b.  Pace  of  Course  (appropriate  pace  or  course  length,  appropriate  amount  of  material 
covered  in  time  frame,  covered  material  quickly). 

c.  VTT  (liked  VTT  instruction,  technology,  or  using  VTT  system). 

d.  Instructors  (knowledge,  helpfulness,  teaching  style,  efficient). 

e.  Exercises  and  Practice  (real  plotting,  hands-on). 

f.  Homework  and  Calculations  (strip  form  computations,  repetition  helped  learn, 

problem  solving,  learned  new  methods,  challenging). 

g.  Instructional  Strategies  (step  by  step  presentation,  practical  application, 
course  organization,  feedback,  clarifying  how  to  use  strip  forms). 

h.  Instructional  Materials  (video,  handouts,  visuals  of  strip  forms,  publications, 
revised  strip  forms). 

i.  Interaction  (with  instructors  and  distant  students). 

j.  Miscellaneous  (facilitator  helped,  small  class  size,  open  classroom,  intense  learning). 
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Table  E-4 


Percentage  of  Treatment  Group  Responses  to  Open-Ended  Question 
"What  did  you  like  the  least  about  this  course?" 


Treatment  Group 


Group  and  Comment  Category 

Traditional 

Local 

Remote 

All  Groups 

a.  Subject  Matter 

3.8 

3.0 

0.0 

2.4 

b.  Pace  of  Course  Too  Fast 

11.3 

10.6 

17.2 

12.5 

c.  VTT 

0.0 

6.8 

4.7 

4.8 

d.  Instructors 

0.0 

4.5 

12.5 

5.6 

e.  Exercises  and  Practice 

5.7 

0.8 

1.6 

2.0 

f.  Homework  and  Calculations 

18.9 

19.7 

15.6 

18.5 

g.  Instructional  Strategies 

3.7 

0.8 

1.6 

1.6 

h.  Instructional  Materials 

3.7 

2.3 

3.1 

2.8 

i.  Facilities 

0.0 

11.4 

0.0 

6.0 

j.  Limited  Access 

1.9 

3.8 

1.6 

2.8 

k.  Facilitators 

0.0 

0.0 

3.1 

0.8 

1.  Miscellaneous 

5.7 

4.5 

7.8 

5.6 

No  Comment 

45.3 

31.8 

31.2 

34.6 

Total  Percent 

100.0 

100.0 

100.0 

100.0 

Number  of  1st  Comments 

28 

86 

41 

155 

Number  of  2nd  Comments 

1 

4 

3 

8 

Total  Comments 

29 

90 

44 

163 

Number  No  Comment 

24 

42 

20 

86 

Total  Comments  and  No  Comment 

53 

132 

64 

249 

Number  of  Students 

52 

128 

61 

241 

Note:  Divisor  for  percents  was  total  of  comments  and  no  comment 

a.  Subject  Matter  (too  hard,  more  theory,  explain  terminology,  pay  attention  to  every  detail). 

b.  Pace  of  Course  (too  fast,  course  too  short). 

c.  VTT  (delays,  technical  difficulties,  using  microphone  to  talk,  could  not  see  other  students). 

d.  Instructors  (knowledge  of  some  topics,  use  of  VTT  system,  getting  help,  teaching  style). 

e.  Exercises  and  Practice  (not  enough  plotting  or  hands-on,  no  actual  shooting  using  sextant). 

f.  Homework  and  Calculations  (repetitive  calculations,  too  much  or  too  difficult, 

concentrated  in  a  few  days,  not  realistic,  no  chance  to  ask  questions  on  it). 

g.  Instructional  Strategies  (organization,  limited  practical  applications,  no  computer  use). 

h.  Instructional  Materials  (video  outdated,  not  interesting). 

i.  Facilities  (small  tables,  room  too  small,  room  too  warm). 

j.  Limited  Access  to  Classroom  (to  do  homework  during  or  after  class,  closed  too  early). 

k.  Facilitators  (lack  of  subject  knowledge). 

l.  Miscellaneous  (dress  requirements,  travel,  too  slow,  test  answers,  more  course  offerings, 

slow  students  need  prerequisite  knowledge,  emphasis  on  new  strip  forms). 
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Table  E-5 


Percentage  of  Treatment  Group  Responses  to  Open-Ended  Question 
"Discuss  any  suggestions  you  have  for  improving  the  course" 


Treatment  Group 


Group  and  Comment  Category 

Traditional 

Local 

Remote 

All  Groups 

a.  Subject  Matter 

7.3 

5.8 

2.9 

5.3 

b.  Pace  of  Course  Slower 

5.5 

6.5 

7.3 

6.5 

c.  VTT 

0.0 

5.1 

4.3 

3.8 

d.  Instructors 

0.0 

1.5 

8.7 

3.1 

e.  Exercises  and  Practice 

9.1 

3.6 

2.9 

4.6 

f.  Instructional  Strategies 

12.7 

10.9 

5.8 

9.9 

g.  Instructional  Materials 

7.3 

2.9 

13.0 

6.5 

h.  Facilities 

0.0 

8.7 

1.5 

4.9 

i.  Facilitators 

0.0 

0.7 

13.0 

3.8 

j.  Access  to  Room 

3.6 

3.6 

0.0 

2.7 

k.  Miscellaneous 

1.8 

4.3 

5.8 

4.2 

No  Comment 

52.7 

46.4 

34.8 

44.7 

Total  Percent 

100.0 

100.0 

100.0 

100.0 

Number  of  1st  Comments 

23 

64 

37 

124 

Number  of  2nd  Comments 

3 

9 

7 

19 

Number  of  3rd  Comments 

0 

1 

1 

2 

Total  Comments 

26 

74 

45 

145 

Number  No  Comment 

29 

64 

24 

117 

Total  Comments  and  No  Comment 

55 

138 

69 

262 

Number  of  Students 

52 

128 

61 

241 

Note:  Divisor  for  percents  was  total  of  comments  and  no  comment 

a.  Subject  Matter  (more  theory,  readings,  or  explain  reference  pubs,  lifeboat  navigation). 

b.  Pace  of  Course  (slower  or  increase  course  length). 

c.  VTT  (audio,  non-glare  monitors,  view  of  local  students  for  remote  site,  sound  activated 
microphones,  computer  model  showing  hour  angles,  prefer  traditional  class). 

d.  Instmctor  (assist  students,  help  with  homework,  avoid  redundancy,  available  after  class). 

e.  Exercises  and  Practice  (more  plotting,  hands-on  work  shooting  with  sextant). 

f.  Instructional  Strategies  (order  of  topics,  plot  sooner,  more  on  plotting  rales,  less  homework, 

review  homework,  homework  checksum,  pace  to  student  ability,  new  celestial  technology). 

g.  Instructional  Materials  (extra  strip  forms  for  practice,  better  plotting  tools, 
new  videotape,  separate  homework  and  notetaking  sheets,  reference  books). 

h.  Facilities  (larger  tables,  more  work  space,  adjust  room  temperature). 

i.  Facilitators  (more  knowledge  of  material,  knowledgeable  person  for  remote  site). 

j.  Access  to  Classroom  (more  access  to  classroom  for  doing  homework). 

k.  Miscellaneous  (relate  to  Coast  Guard,  grade  less  on  plotting,  basic  course  as  prerequisite, 
expand  VTT,  faster  pace,  use  PIP  view,  pass  skill  to  shipmates,  piloting-magnetic  topic). 
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Appendix  F 

Interaction  Tally  Summary  Statistics 
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Table  F-1 

Interaction  Tally  Category  Numbers  Used  in  Tables 


Instructor  Question  Categories  (1,  2,  3)  answered  from  either  Local  or  Remote  site: 

(1)  Open  to  Any  Site 

(2)  Names  a  Site 

(3)  Specific  Student 

Smdent  Questions  Categories  (4,  5)  from  either  Local  or  Remote  Site: 

(4)  Student  Initiated 

(5)  Conversations 

Other  Categories  (6  and  7  combine  Local  and  Remote)(8  for  Local  Only): 

(6)  No  Answer  to  Instructor  Question 

(7)  No  Answer  to  Instructor  Query  for  "Any  Questions" 

(8)  Instructor  Reminds  Local  Student  to  Use  Microphone 


Note:  Definitions  for  these  categories  are  given  in  the  method  section. 


Table  F-2 

Correlation  Between  Observer  Interaction  Tallies 


Number  of 


Interaction  Category 

Observations 

Correlation 

(1)  Instructor  Question  Open  to  Any  Site 

6 

0.939 

(2)  Instructor  Question  Names  a  Site 

6 

0.981 

(3)  Instructor  Questions  Specific  Student 

6 

0.980 

(4)  Student  Initiated  Questions 

6 

0.993 

(5)  Student  Conversations  and  Exchanges 

6 

0.667 

Combined  Over  All  5  Categories 

30 

0.975 

Explanation  of  Table  F-2:  Rater  agreement  using  the  interaction  tally  was 
recorded  during  three  half  day  periods  (two  afternoon  sessions  and  one  morn¬ 
ing  session).  Pearson  correlation  coefficients  were  calculated  for  two  raters 
observing  the  same  sessions  in  which  there  was  one  local  and  one  remote  site 
(perfect  correspondence  between  raters  would  be  indicated  by  a  coefficient  of 
+1.0).  The  0.975  correlation  between  rater  interaction  frequencies  was  based 
on  an  overall  total  of  30  pairs  of  observations  where  each  rater  made  observa¬ 
tions  for  five  tally  categories  by  two  sites  by  three  recording  periods.  Of  414 
total  interactions,  one  rater  recorded  209  and  the  other  recorded  205.  The 
correlations  for  the  individual  categories  were  based  on  six  observations  in 
each  category  (i.e.,  two  sites  by  three  recording  periods  for  each  rater). 
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Table  F-3 

Interaction  Frequencies  and  Percentages  by  Individual  Category 


Category 

Frequency 

Percent  of  All 
Categories  1  to  8 

Percent  of  Only 
Categories  1  to  5 

Instructor  Questions  (1  to  3): 

(1)  Open  to  Any  Site 

Local 

514 

14.9% 

18.1% 

Remote 

392 

11.4% 

13.8% 

Subtotal  Local  +  Remote 

906 

26.4% 

31.8% 

(2)  Names  a  Site 

Local 

123 

3.6% 

4.3% 

Remote 

440 

12.8% 

15.5% 

Subtotal  Local  +  Remote 

563 

16.4% 

19.8% 

(3)  Specific  Student 

Local 

494 

14.4% 

17.3% 

Remote 

255 

7.4% 

8.9% 

Subtotal  Local  +  Remote 

749 

21.8% 

26.3% 

Student  Questions  (4  and  5): 

(4)  Student  Initiated 

Local 

296 

8.6% 

10.4% 

Remote 

160 

4.7% 

5.6% 

Subtotal  Local  +  Remote 

456 

13.3% 

16.0% 

(5)  Conversations/Exchanges 

Local 

79 

2.3% 

2.8% 

Remote 

94 

2.7% 

3.3% 

Subtotal  Local  +  Remote 

173 

5.1% 

6.1% 

Subtotal  Categories  (1  to  5) 

2^ 

83% 

100% 

Other  Categories  (6  to  8): 

(6)  No  Answer  to 

Instructor  Question 

132 

3.9% 

(7)  No  Answer  to  Instructor 

"Any  Questions"  Query 

334 

9.7% 

(8)  Microphone  Reminders 

to  Local  Students 

115 

3.4% 

Grand  Total  Categories  (1  to  8) 

3428 

100% 

Frequencies  are  sums  for  six  classes.  Percents  may  not  add  to  100%  due  to  rounding. 
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Table  F-4 

Interaction  Frequencies  and  Percentages  by  Combined  Category 


Percent  of  All 

Percent  of  Only 

Category 

Frequency 

Categories  1  to  8 

Categories  1  to  5 

Instructor  Questions  (1+2+3)  Combined 

Local 

1131 

33.0% 

39.7% 

Remote 

1087 

31.7% 

38.1% 

Subtotal  Local  +  Remote 

2218 

64.7% 

77.9% 

Student  Questions  (4+5)  Combined 

Local 

375 

10.9% 

13.2% 

Remote 

254 

7.4% 

8.9% 

Subtotal  Local  +  Remote 

629 

18.3% 

22.1% 

Subtotal  Categories  (1  to  5) 

2847 

83% 

100% 

No  Answer  to  Instructor  Question  (6) 

132 

3.9% 

No  Answer  to  "Any  Questions"  (7) 

334 

9.7% 

Microphone  Reminders  to  Locals  (8) 

115 

3.4% 

Grand  Total  Categories  (1  to  8) 

3428 

100% 

Combined  Instructor  +  Student  Questions  (1+2+3+4+5) 

Local 

1506 

— 

52.8% 

Remote 

1341 

— 

47.1% 

Subtotal  Categories  (1  to  5) 

2847 

100% 

Frequencies  are  sums  for  six  classes.  Percents  may  not  add  to  100%  due  to  rounding. 


Table  F-5 

Symbols  and  Formulas  Used  in  Computations  of  Ratios  and  Rates 


I 

=r 

Sum  of  number  of  interactions  over  six  classes 

H 

= 

Total  recording  hours  over  six  classes 

S 

= 

Total  number  of  students  over  six  classes 

c 

Number  of  classes 

I/s 

= 

Interactions  per  student 

S/C 

Students  per  class 

I/H 

= 

Interactions  per  hour 

(I/H)/(S/C) 

= 

Interactions  per  hour  per  student 

(I/H)/(S/C) 

= 

(FS)/(H/C)  =  [(I/C)/(H/C)]/(S/C) 
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Table  F-6 

Computation  of  Instructor  and  Student  Question  Ratios 


Categories 

1+2+3 

Instructor 

Questions 

Categories 

4+5 

Student 

Questions 

Categories 
1+2+3+4+5 
Instructor 
&  Student 

Measure 

Local 

Remote 

Local 

Remote 

Local 

Remote 

Sum  of  Interactions  (I) 

1131 

1087 

375 

254 

1506 

1341 

Total  Students  (S) 

101 

46 

101 

46 

101 

46 

Number  of  Classes  (C) 

6 

6 

6 

6 

6 

6 

Students  Per  Class  (S/C) 

16.83 

7.66 

16.83 

7.66 

16.83 

7.66 

Total  Recording  Hours  (H) 

53.016 

53.016 

53.016 

53.016 

53.016 

53.016 

Interaction  Ratios: 

Per  Student  (I/S) 

11.19 

23.63 

3.71 

5.52 

14.91 

29.15 

Per  Hour  (I/H) 

21.33 

20.50 

7.07 

4.79 

28.40 

25.29 

Per  Hour  Per  Student 

1.267 

2.674 

0.420 

0.624 

1.687 

3.299 

[(I/H)/(S/C)] 


Table  F-7 

Computation  of  Interaction  Ratios  for  Other  Categories  (6, 7,  8) 


Measure 

Category  6 

No  Answer 

To  Question 

Category  7 

No  Answer 

To  "Any 
Questions?" 

Category  8 
Microphone 
Reminders 
to  Locals 

Sum  of  Interactions  (I) 

132 

334 

115 

Total  Students  (S) 

147 

147 

101 

Number  of  Classes  (C) 

6 

6 

6 

Students  Per  Class  (S/C) 

24.5 

24.5 

16.83 

Total  Recording  Hours  (H) 

53.016 

53.016 

53.016 

Interaction  Ratios: 

Per  Student  (I/S) 

0.897 

2.272 

1.1386 

Per  Hour  (I/H) 

2.489 

6.299 

2.1691 

Per  Hour  Per  Student 
[(I/H)/(S/C)] 

0.101 

0.257 

0.128 
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Table  F-8 

Interaction  Rates  by  Category  for  Local  and  Remote 


Per  Hour 


Per  Hour  Per  Student 

(I/H)  (im)/(S/C) 


Category 

Local 

Remote 

Local 

Remote 

Instructor  Questions: 

(1)  Open  to  Any  Site 

9.69 

7.39 

0.57 

0.96 

(2)  Names  a  Site 

2.32 

8.29 

0.13 

1.08 

(3)  Specific  Student 

9.31 

4.80 

0.55 

0.62 

Student  Questions: 

(4)  Student  Initiated 

5.58 

3.01 

0.33 

0.39 

(5)  Conversations  and  Exchanges 

1.49 

1.77 

0.08 

0.23 

Table  F-9 

Interaction  Rates  by  Days  by  Morning  (AM)  and  Afternoon  (PM) 


Measure  and  Category 


Total  Recording  Hours 

Interactions  Per  Hour  (I/H) 

Instructor  and  Student 
(Categories  1+2+3+4+5) 

No  Answer  to  Instructor  Question  (6) 
No  Answer  to  "Any  Questions"  (7) 
Microphone  Reminders  to  Locals  (8) 


Day  1  Day  2  Day  3 


AM 

PM 

AM 

PM 

AM 

PM 

11.01 

7.07 

13.95 

7.74 

8.59 

4.85 

40.96 

55.02 

54.76 

51.03 

66.23 

66.59 

5.35 

3.11 

1.43 

1.16 

1.74 

1.03 

5.90 

4.66 

5.80 

5.81 

8.14 

8.24 

2.99 

2.40 

2.07 

2.06 

1.51 

1.44 

Interactions  Per  Hour  Per  Student  (I/H)/(S/C) 

Instructor  and  Student 
(Categories  1+2+3+4+5) 

No  Answer  to  Instructor  Question  (6) 

No  Answer  to  "Any  Questions"  (7) 
Microphone  Reminders  to  Locals  (8) 


1.67 

2.24 

2.23 

2.08 

2.70 

2.71 

0.22 

0.13 

0.06 

0.05 

0.07 

0.04 

0.24 

0.19 

0.24 

0.24 

0.33 

0.34 

0.17 

0.14 

0.12 

0.12 

0.09 

0.08 
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Table  F-10 

Interaction  Rates  by  Days 


Interactions  Per  Hour  (I/H) 

Day  1 

Day  2 

Day  3 

Instructor  Questions  (Categories  1+2+3) 

Local: 

19.72 

20.52 

27.81 

Remote: 

19.17 

18.61 

23.88 

Combined  Local  and  Remote: 

38.89 

39.14 

52.43 

Student  Questions  (Categories  4+5) 

Local: 

4.08 

9.33 

7.49 

Remote: 

3.42 

5.08 

6.15 

Combined  Local  and  Remote: 

7.51 

14.41 

13.64 

Other  Categories  (6,  7,  8) 

No  Answer  to  Instructor  Question  (6) 

4.53 

1.30 

1.48 

No  Answer  to  "Any  Questions"  Query  (7) 

5.41 

5.87 

8.15 

Microphone  Reminders  to  Local  Students  (8) 

2.76 

2.09 

1.48 

Interactions  Per  Hour  Per  Student  (I/H)/(S/C) 

Day  1 

Day  2 

Day  3 

Instructor  Questions  (Categories  1+2+3) 

Local: 

1.17 

1.21 

1.66 

Remote: 

2.50 

2.48 

3.28 

Combined  Local  and  Remote: 

1.58 

1.60 

2.16 

Student  Questions  (Categories  4+5) 

Local: 

0.24 

0.55 

0.44 

Remote: 

0.44 

0.67 

0.82 

Combined  Local  and  Remote: 

0.30 

0.59 

0.56 

Other  Categories  (6,  7,  8) 

No  Answer  to  Instructor  Question  (6) 

0.18 

0.05 

0.06 

No  Answer  to  "Any  Questions"  Query  (7) 

0.22 

0.23 

0.33 

Microphone  Reminders  to  Local  Students  (8) 

0.16 

0.12 

0.08 
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